You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2017/01/31 21:53:13 UTC
[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3809
- Still Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3809/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC
5 tests failed.
FAILED: org.apache.solr.cloud.ReplaceNodeTest.test
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([321AC469C3E8746B:BA4EFBB36D141993]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.ReplaceNodeTest.test(ReplaceNodeTest.java:79)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.SimpleCollectionCreateDeleteTest.test
Error Message:
java.io.IOException: Couldn't instantiate org.apache.zookeeper.ClientCnxnSocketNIO
Stack Trace:
org.apache.solr.common.SolrException: java.io.IOException: Couldn't instantiate org.apache.zookeeper.ClientCnxnSocketNIO
at __randomizedtesting.SeedInfo.seed([321AC469C3E8746B:BA4EFBB36D141993]:0)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:170)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:116)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:111)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:98)
at org.apache.solr.cloud.AbstractDistribZkTestBase.printLayout(AbstractDistribZkTestBase.java:322)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.distribTearDown(AbstractFullDistribZkTestBase.java:1500)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:962)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Couldn't instantiate org.apache.zookeeper.ClientCnxnSocketNIO
at org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:1782)
at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:447)
at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:380)
at org.apache.solr.common.cloud.SolrZooKeeper.<init>(SolrZooKeeper.java:43)
at org.apache.solr.common.cloud.ZkClientConnectionStrategy.createSolrZooKeeper(ZkClientConnectionStrategy.java:105)
at org.apache.solr.common.cloud.DefaultConnectionStrategy.connect(DefaultConnectionStrategy.java:37)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:145)
... 37 more
Caused by: java.io.IOException: Too many open files
at sun.nio.ch.IOUtil.makePipe(Native Method)
at sun.nio.ch.KQueueSelectorImpl.<init>(KQueueSelectorImpl.java:84)
at sun.nio.ch.KQueueSelectorProvider.openSelector(KQueueSelectorProvider.java:42)
at java.nio.channels.Selector.open(Selector.java:227)
at org.apache.zookeeper.ClientCnxnSocketNIO.<init>(ClientCnxnSocketNIO.java:43)
at sun.reflect.GeneratedConstructorAccessor218.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:1779)
... 43 more
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.SimpleCollectionCreateDeleteTest
Error Message:
47 threads leaked from SUITE scope at org.apache.solr.cloud.SimpleCollectionCreateDeleteTest: 1) Thread[id=14909, name=Thread-3301, state=WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at java.lang.Object.wait(Native Method) at java.lang.Thread.join(Thread.java:1249) at java.lang.Thread.join(Thread.java:1323) at org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:302) at org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:309) at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:490) 2) Thread[id=14954, name=Connection evictor, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:745) 3) Thread[id=14960, name=qtp2115518317-14960-acceptor-0@79f0ebf7-ServerConnector@3ebf7f1e{SSL,[ssl, http/1.1]}{127.0.0.1:62155}, state=RUNNABLE, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:373) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:745) 4) Thread[id=14978, name=zkCallback-2757-thread-1, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 5) Thread[id=14942, name=zkCallback-2748-thread-1, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 6) Thread[id=14953, name=zkCallback-2748-thread-2, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 7) Thread[id=14944, name=OverseerCollectionConfigSetProcessor-97379784573845509-127.0.0.1:62149_-n_0000000000, state=TIMED_WAITING, group=Overseer collection creation process.] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.apache.solr.cloud.DistributedQueue.peekElements(DistributedQueue.java:343) at org.apache.solr.cloud.OverseerTaskQueue.peekTopN(OverseerTaskQueue.java:235) at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:227) at java.lang.Thread.run(Thread.java:745) 8) Thread[id=14976, name=TEST-SimpleCollectionCreateDeleteTest.test-seed#[321AC469C3E8746B]-SendThread(127.0.0.1:62145), state=RUNNABLE, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) 9) Thread[id=14928, name=qtp832258634-14928, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 10) Thread[id=14936, name=Connection evictor, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:745) 11) Thread[id=14910, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:178) at java.lang.Thread.run(Thread.java:745) 12) Thread[id=14931, name=org.eclipse.jetty.server.session.HashSessionManager@31473cfTimer, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 13) Thread[id=14947, name=Thread-3310, state=WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1459) 14) Thread[id=14956, name=TEST-SimpleCollectionCreateDeleteTest.test-seed#[321AC469C3E8746B]-SendThread(127.0.0.1:62145), state=RUNNABLE, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) 15) Thread[id=14979, name=Connection evictor, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:745) 16) Thread[id=14952, name=coreZkRegister-7658-thread-1, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 17) Thread[id=14972, name=Connection evictor, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:745) 18) Thread[id=14977, name=TEST-SimpleCollectionCreateDeleteTest.test-seed#[321AC469C3E8746B]-EventThread, state=WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494) 19) Thread[id=14929, name=qtp832258634-14929, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 20) Thread[id=14913, name=ProcessThread(sid:0 cport:-1):, state=WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:120) 21) Thread[id=14961, name=qtp2115518317-14961, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 22) Thread[id=14967, name=org.eclipse.jetty.server.session.HashSessionManager@2a9873e6Timer, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 23) Thread[id=14945, name=OverseerHdfsCoreFailoverThread-97379784573845509-127.0.0.1:62149_-n_0000000000, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:139) at java.lang.Thread.run(Thread.java:745) 24) Thread[id=14959, name=qtp2115518317-14959, state=RUNNABLE, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:745) 25) Thread[id=14965, name=qtp2115518317-14965, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 26) Thread[id=14951, name=searcherExecutor-7666-thread-1, state=WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 27) Thread[id=14926, name=qtp832258634-14926, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 28) Thread[id=14955, name=Connection evictor, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:745) 29) Thread[id=14966, name=qtp2115518317-14966, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 30) Thread[id=14940, name=TEST-SimpleCollectionCreateDeleteTest.test-seed#[321AC469C3E8746B]-SendThread(127.0.0.1:62145), state=RUNNABLE, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) 31) Thread[id=14964, name=qtp2115518317-14964, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 32) Thread[id=14923, name=qtp832258634-14923, state=RUNNABLE, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:745) 33) Thread[id=14948, name=coreContainerWorkExecutor-7659-thread-1, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 34) Thread[id=14924, name=qtp832258634-14924-acceptor-0@66b2406a-ServerConnector@2b0300e9{SSL,[ssl, http/1.1]}{127.0.0.1:62149}, state=RUNNABLE, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:373) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:745) 35) Thread[id=14941, name=TEST-SimpleCollectionCreateDeleteTest.test-seed#[321AC469C3E8746B]-EventThread, state=WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494) 36) Thread[id=14949, name=watches-2749-thread-1, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 37) Thread[id=14958, name=zkCallback-2752-thread-1, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 38) Thread[id=14935, name=Connection evictor, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:745) 39) Thread[id=14963, name=qtp2115518317-14963, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 40) Thread[id=14943, name=OverseerStateUpdate-97379784573845509-127.0.0.1:62149_-n_0000000000, state=TIMED_WAITING, group=Overseer state updater.] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:164) at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:138) at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:179) at java.lang.Thread.run(Thread.java:745) 41) Thread[id=14957, name=TEST-SimpleCollectionCreateDeleteTest.test-seed#[321AC469C3E8746B]-EventThread, state=WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494) 42) Thread[id=14971, name=Connection evictor, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:745) 43) Thread[id=14930, name=qtp832258634-14930, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 44) Thread[id=14927, name=qtp832258634-14927, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 45) Thread[id=14911, name=SessionTracker, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at java.lang.Object.wait(Native Method) at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146) 46) Thread[id=14925, name=qtp832258634-14925, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 47) Thread[id=14962, name=qtp2115518317-14962, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 47 threads leaked from SUITE scope at org.apache.solr.cloud.SimpleCollectionCreateDeleteTest:
1) Thread[id=14909, name=Thread-3301, state=WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1249)
at java.lang.Thread.join(Thread.java:1323)
at org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:302)
at org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:309)
at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:490)
2) Thread[id=14954, name=Connection evictor, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:745)
3) Thread[id=14960, name=qtp2115518317-14960-acceptor-0@79f0ebf7-ServerConnector@3ebf7f1e{SSL,[ssl, http/1.1]}{127.0.0.1:62155}, state=RUNNABLE, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:373)
at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:745)
4) Thread[id=14978, name=zkCallback-2757-thread-1, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
5) Thread[id=14942, name=zkCallback-2748-thread-1, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
6) Thread[id=14953, name=zkCallback-2748-thread-2, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
7) Thread[id=14944, name=OverseerCollectionConfigSetProcessor-97379784573845509-127.0.0.1:62149_-n_0000000000, state=TIMED_WAITING, group=Overseer collection creation process.]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.apache.solr.cloud.DistributedQueue.peekElements(DistributedQueue.java:343)
at org.apache.solr.cloud.OverseerTaskQueue.peekTopN(OverseerTaskQueue.java:235)
at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:227)
at java.lang.Thread.run(Thread.java:745)
8) Thread[id=14976, name=TEST-SimpleCollectionCreateDeleteTest.test-seed#[321AC469C3E8746B]-SendThread(127.0.0.1:62145), state=RUNNABLE, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
9) Thread[id=14928, name=qtp832258634-14928, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
10) Thread[id=14936, name=Connection evictor, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:745)
11) Thread[id=14910, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:178)
at java.lang.Thread.run(Thread.java:745)
12) Thread[id=14931, name=org.eclipse.jetty.server.session.HashSessionManager@31473cfTimer, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
13) Thread[id=14947, name=Thread-3310, state=WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at org.apache.solr.core.CloserThread.run(CoreContainer.java:1459)
14) Thread[id=14956, name=TEST-SimpleCollectionCreateDeleteTest.test-seed#[321AC469C3E8746B]-SendThread(127.0.0.1:62145), state=RUNNABLE, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
15) Thread[id=14979, name=Connection evictor, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:745)
16) Thread[id=14952, name=coreZkRegister-7658-thread-1, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
17) Thread[id=14972, name=Connection evictor, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:745)
18) Thread[id=14977, name=TEST-SimpleCollectionCreateDeleteTest.test-seed#[321AC469C3E8746B]-EventThread, state=WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
19) Thread[id=14929, name=qtp832258634-14929, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
20) Thread[id=14913, name=ProcessThread(sid:0 cport:-1):, state=WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:120)
21) Thread[id=14961, name=qtp2115518317-14961, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
22) Thread[id=14967, name=org.eclipse.jetty.server.session.HashSessionManager@2a9873e6Timer, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
23) Thread[id=14945, name=OverseerHdfsCoreFailoverThread-97379784573845509-127.0.0.1:62149_-n_0000000000, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:139)
at java.lang.Thread.run(Thread.java:745)
24) Thread[id=14959, name=qtp2115518317-14959, state=RUNNABLE, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:745)
25) Thread[id=14965, name=qtp2115518317-14965, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
26) Thread[id=14951, name=searcherExecutor-7666-thread-1, state=WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
27) Thread[id=14926, name=qtp832258634-14926, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
28) Thread[id=14955, name=Connection evictor, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:745)
29) Thread[id=14966, name=qtp2115518317-14966, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
30) Thread[id=14940, name=TEST-SimpleCollectionCreateDeleteTest.test-seed#[321AC469C3E8746B]-SendThread(127.0.0.1:62145), state=RUNNABLE, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
31) Thread[id=14964, name=qtp2115518317-14964, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
32) Thread[id=14923, name=qtp832258634-14923, state=RUNNABLE, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:745)
33) Thread[id=14948, name=coreContainerWorkExecutor-7659-thread-1, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
34) Thread[id=14924, name=qtp832258634-14924-acceptor-0@66b2406a-ServerConnector@2b0300e9{SSL,[ssl, http/1.1]}{127.0.0.1:62149}, state=RUNNABLE, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:373)
at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:745)
35) Thread[id=14941, name=TEST-SimpleCollectionCreateDeleteTest.test-seed#[321AC469C3E8746B]-EventThread, state=WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
36) Thread[id=14949, name=watches-2749-thread-1, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
37) Thread[id=14958, name=zkCallback-2752-thread-1, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
38) Thread[id=14935, name=Connection evictor, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:745)
39) Thread[id=14963, name=qtp2115518317-14963, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
40) Thread[id=14943, name=OverseerStateUpdate-97379784573845509-127.0.0.1:62149_-n_0000000000, state=TIMED_WAITING, group=Overseer state updater.]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:164)
at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:138)
at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:179)
at java.lang.Thread.run(Thread.java:745)
41) Thread[id=14957, name=TEST-SimpleCollectionCreateDeleteTest.test-seed#[321AC469C3E8746B]-EventThread, state=WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
42) Thread[id=14971, name=Connection evictor, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:745)
43) Thread[id=14930, name=qtp832258634-14930, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
44) Thread[id=14927, name=qtp832258634-14927, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
45) Thread[id=14911, name=SessionTracker, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at java.lang.Object.wait(Native Method)
at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)
46) Thread[id=14925, name=qtp832258634-14925, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
47) Thread[id=14962, name=qtp2115518317-14962, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([321AC469C3E8746B]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.SimpleCollectionCreateDeleteTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=14952, name=coreZkRegister-7658-thread-1, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2) Thread[id=14967, name=org.eclipse.jetty.server.session.HashSessionManager@2a9873e6Timer, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 3) Thread[id=14959, name=qtp2115518317-14959, state=RUNNABLE, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.nio.ch.IOUtil.drain(Native Method) at sun.nio.ch.KQueueSelectorImpl.updateSelectedKeys(KQueueSelectorImpl.java:181) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:122) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:745) 4) Thread[id=14951, name=searcherExecutor-7666-thread-1, state=WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 5) Thread[id=14978, name=zkCallback-2757-thread-1, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 6) Thread[id=14940, name=TEST-SimpleCollectionCreateDeleteTest.test-seed#[321AC469C3E8746B]-SendThread(127.0.0.1:62145), state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003) 7) Thread[id=14942, name=zkCallback-2748-thread-1, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 8) Thread[id=14923, name=qtp832258634-14923, state=RUNNABLE, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.nio.ch.KQueueArrayWrapper.interrupt(Native Method) at sun.nio.ch.KQueueArrayWrapper.interrupt(KQueueArrayWrapper.java:203) at sun.nio.ch.KQueueSelectorImpl.wakeup(KQueueSelectorImpl.java:257) at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:213) at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:219) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:116) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:745) 9) Thread[id=14948, name=coreContainerWorkExecutor-7659-thread-1, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 10) Thread[id=14953, name=zkCallback-2748-thread-2, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 11) Thread[id=14976, name=TEST-SimpleCollectionCreateDeleteTest.test-seed#[321AC469C3E8746B]-SendThread(127.0.0.1:62145), state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003) 12) Thread[id=14949, name=watches-2749-thread-1, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 13) Thread[id=14958, name=zkCallback-2752-thread-1, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 14) Thread[id=14931, name=org.eclipse.jetty.server.session.HashSessionManager@31473cfTimer, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 15) Thread[id=14947, name=Thread-3310, state=WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1459) 16) Thread[id=14956, name=TEST-SimpleCollectionCreateDeleteTest.test-seed#[321AC469C3E8746B]-SendThread(127.0.0.1:62145), state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=14952, name=coreZkRegister-7658-thread-1, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
at sun.misc.Unsafe.park(Native Me
[...truncated too long message...]
eeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] > 7) Thread[id=14942, name=zkCallback-2748-thread-1, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 8) Thread[id=14923, name=qtp832258634-14923, state=RUNNABLE, group=TGRP-SimpleCollectionCreateDeleteTest]
[junit4] > at sun.nio.ch.KQueueArrayWrapper.interrupt(Native Method)
[junit4] > at sun.nio.ch.KQueueArrayWrapper.interrupt(KQueueArrayWrapper.java:203)
[junit4] > at sun.nio.ch.KQueueSelectorImpl.wakeup(KQueueSelectorImpl.java:257)
[junit4] > at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:213)
[junit4] > at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:219)
[junit4] > at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:116)
[junit4] > at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
[junit4] > at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
[junit4] > at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
[junit4] > at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 9) Thread[id=14948, name=coreContainerWorkExecutor-7659-thread-1, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 10) Thread[id=14953, name=zkCallback-2748-thread-2, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 11) Thread[id=14976, name=TEST-SimpleCollectionCreateDeleteTest.test-seed#[321AC469C3E8746B]-SendThread(127.0.0.1:62145), state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] > 12) Thread[id=14949, name=watches-2749-thread-1, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 13) Thread[id=14958, name=zkCallback-2752-thread-1, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 14) Thread[id=14931, name=org.eclipse.jetty.server.session.HashSessionManager@31473cfTimer, state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
[junit4] > at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
[junit4] > at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 15) Thread[id=14947, name=Thread-3310, state=WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
[junit4] > at java.lang.Object.wait(Native Method)
[junit4] > at java.lang.Object.wait(Object.java:502)
[junit4] > at org.apache.solr.core.CloserThread.run(CoreContainer.java:1459)
[junit4] > 16) Thread[id=14956, name=TEST-SimpleCollectionCreateDeleteTest.test-seed#[321AC469C3E8746B]-SendThread(127.0.0.1:62145), state=TIMED_WAITING, group=TGRP-SimpleCollectionCreateDeleteTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] > at __randomizedtesting.SeedInfo.seed([321AC469C3E8746B]:0)
[junit4] Completed [505/691 (3!)] on J0 in 57.25s, 1 test, 3 errors <<< FAILURES!
[...truncated 63044 lines...]
[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3828
- Still unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3828/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC
4 tests failed.
FAILED: org.apache.solr.cloud.DeleteInactiveReplicaTest.deleteInactiveReplicaTest
Error Message:
Deleted core was still loaded!
Stack Trace:
java.lang.AssertionError: Deleted core was still loaded!
at __randomizedtesting.SeedInfo.seed([706A414369DC7549:BD54DAB0267D03AB]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.DeleteInactiveReplicaTest.deleteInactiveReplicaTest(DeleteInactiveReplicaTest.java:86)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.PeerSyncReplicationTest.test
Error Message:
timeout waiting to see all nodes active
Stack Trace:
java.lang.AssertionError: timeout waiting to see all nodes active
at __randomizedtesting.SeedInfo.seed([706A414369DC7549:F83E7E99C72018B1]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.PeerSyncReplicationTest.waitTillNodesActive(PeerSyncReplicationTest.java:326)
at org.apache.solr.cloud.PeerSyncReplicationTest.bringUpDeadNodeAndEnsureNoReplication(PeerSyncReplicationTest.java:277)
at org.apache.solr.cloud.PeerSyncReplicationTest.forceNodeFailureAndDoPeerSync(PeerSyncReplicationTest.java:259)
at org.apache.solr.cloud.PeerSyncReplicationTest.test(PeerSyncReplicationTest.java:138)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.handler.TestReplicationHandler.testRateLimitedReplication
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([706A414369DC7549:F6FE34B6666EF4A4]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.handler.TestReplicationHandler.testRateLimitedReplication(TestReplicationHandler.java:1458)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.handler.TestReqParamsAPI.test
Error Message:
Could not get expected value 'A val' for path 'params/a' full output: { "responseHeader":{ "status":0, "QTime":0}, "params":{ "wt":"json", "useParams":""}, "context":{ "webapp":"/solr", "path":"/dump0", "httpMethod":"GET"}}, from server: http://127.0.0.1:62349/solr/collection1_shard1_replica1
Stack Trace:
java.lang.AssertionError: Could not get expected value 'A val' for path 'params/a' full output: {
"responseHeader":{
"status":0,
"QTime":0},
"params":{
"wt":"json",
"useParams":""},
"context":{
"webapp":"/solr",
"path":"/dump0",
"httpMethod":"GET"}}, from server: http://127.0.0.1:62349/solr/collection1_shard1_replica1
at __randomizedtesting.SeedInfo.seed([706A414369DC7549:F83E7E99C72018B1]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:556)
at org.apache.solr.handler.TestReqParamsAPI.testReqParams(TestReqParamsAPI.java:127)
at org.apache.solr.handler.TestReqParamsAPI.test(TestReqParamsAPI.java:69)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11704 lines...]
[junit4] Suite: org.apache.solr.handler.TestReqParamsAPI
[junit4] 2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReqParamsAPI_706A414369DC7549-001/init-core-data-001
[junit4] 2> 1335440 INFO (SUITE-TestReqParamsAPI-seed#[706A414369DC7549]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 1335444 INFO (SUITE-TestReqParamsAPI-seed#[706A414369DC7549]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN) w/ MAC_OS_X supressed clientAuth
[junit4] 2> 1335445 INFO (SUITE-TestReqParamsAPI-seed#[706A414369DC7549]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReqParamsAPI_706A414369DC7549-001/tempDir-001
[junit4] 2> 1335446 INFO (SUITE-TestReqParamsAPI-seed#[706A414369DC7549]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1335446 INFO (Thread-2338) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1335446 INFO (Thread-2338) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1335556 INFO (SUITE-TestReqParamsAPI-seed#[706A414369DC7549]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:62347
[junit4] 2> 1335588 INFO (jetty-launcher-1630-thread-1) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1335588 INFO (jetty-launcher-1630-thread-2) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1335590 INFO (jetty-launcher-1630-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4ad41493{/solr,null,AVAILABLE}
[junit4] 2> 1335591 INFO (jetty-launcher-1630-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7883e54a{HTTP/1.1,[http/1.1]}{127.0.0.1:62349}
[junit4] 2> 1335591 INFO (jetty-launcher-1630-thread-1) [ ] o.e.j.s.Server Started @1340573ms
[junit4] 2> 1335591 INFO (jetty-launcher-1630-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=62349}
[junit4] 2> 1335592 INFO (jetty-launcher-1630-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@657e64b9{/solr,null,AVAILABLE}
[junit4] 2> 1335592 ERROR (jetty-launcher-1630-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1335592 INFO (jetty-launcher-1630-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3aff7384{HTTP/1.1,[http/1.1]}{127.0.0.1:62350}
[junit4] 2> 1335592 INFO (jetty-launcher-1630-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 1335592 INFO (jetty-launcher-1630-thread-2) [ ] o.e.j.s.Server Started @1340574ms
[junit4] 2> 1335593 INFO (jetty-launcher-1630-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1335594 INFO (jetty-launcher-1630-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1335593 INFO (jetty-launcher-1630-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=62350}
[junit4] 2> 1335594 INFO (jetty-launcher-1630-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-13T00:04:40.288Z
[junit4] 2> 1335594 ERROR (jetty-launcher-1630-thread-2) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1335594 INFO (jetty-launcher-1630-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 1335594 INFO (jetty-launcher-1630-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1335594 INFO (jetty-launcher-1630-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1335594 INFO (jetty-launcher-1630-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-13T00:04:40.288Z
[junit4] 2> 1335597 INFO (jetty-launcher-1630-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1335597 INFO (jetty-launcher-1630-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1335614 INFO (jetty-launcher-1630-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:62347/solr
[junit4] 2> 1335619 INFO (jetty-launcher-1630-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:62347/solr
[junit4] 2> 1335682 INFO (jetty-launcher-1630-thread-1) [n:127.0.0.1:62349_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1335684 INFO (jetty-launcher-1630-thread-2) [n:127.0.0.1:62350_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1335685 INFO (jetty-launcher-1630-thread-1) [n:127.0.0.1:62349_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:62349_solr
[junit4] 2> 1335686 INFO (jetty-launcher-1630-thread-2) [n:127.0.0.1:62350_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:62350_solr
[junit4] 2> 1335687 INFO (jetty-launcher-1630-thread-1) [n:127.0.0.1:62349_solr ] o.a.s.c.Overseer Overseer (id=97448380343451653-127.0.0.1:62349_solr-n_0000000000) starting
[junit4] 2> 1335690 INFO (zkCallback-1640-thread-1-processing-n:127.0.0.1:62349_solr) [n:127.0.0.1:62349_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1335690 INFO (zkCallback-1641-thread-1-processing-n:127.0.0.1:62350_solr) [n:127.0.0.1:62350_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1335707 INFO (jetty-launcher-1630-thread-1) [n:127.0.0.1:62349_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:62349_solr
[junit4] 2> 1335710 INFO (zkCallback-1640-thread-1-processing-n:127.0.0.1:62349_solr) [n:127.0.0.1:62349_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1335710 INFO (zkCallback-1641-thread-1-processing-n:127.0.0.1:62350_solr) [n:127.0.0.1:62350_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1335784 INFO (jetty-launcher-1630-thread-2) [n:127.0.0.1:62350_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReqParamsAPI_706A414369DC7549-001/tempDir-001/node2/.
[junit4] 2> 1335784 INFO (jetty-launcher-1630-thread-1) [n:127.0.0.1:62349_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReqParamsAPI_706A414369DC7549-001/tempDir-001/node1/.
[junit4] 2> 1336638 INFO (SUITE-TestReqParamsAPI-seed#[706A414369DC7549]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1336640 INFO (SUITE-TestReqParamsAPI-seed#[706A414369DC7549]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:62347/solr ready
[junit4] 2> 1336664 INFO (qtp1790213708-10273) [n:127.0.0.1:62349_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf1&name=collection1&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1336668 INFO (OverseerThreadFactory-4851-thread-1-processing-n:127.0.0.1:62349_solr) [n:127.0.0.1:62349_solr ] o.a.s.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 1336781 INFO (qtp1790213708-10274) [n:127.0.0.1:62349_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=collection1_shard1_replica1&action=CREATE&numShards=1&collection=collection1&shard=shard1&wt=javabin&version=2
[junit4] 2> 1336786 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=collection1_shard1_replica2&action=CREATE&numShards=1&collection=collection1&shard=shard1&wt=javabin&version=2
[junit4] 2> 1336902 INFO (zkCallback-1640-thread-1-processing-n:127.0.0.1:62349_solr) [n:127.0.0.1:62349_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1336902 INFO (zkCallback-1641-thread-1-processing-n:127.0.0.1:62350_solr) [n:127.0.0.1:62350_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1337808 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 x:collection1_shard1_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1337808 INFO (qtp1790213708-10274) [n:127.0.0.1:62349_solr c:collection1 s:shard1 x:collection1_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1337817 INFO (qtp1790213708-10274) [n:127.0.0.1:62349_solr c:collection1 s:shard1 x:collection1_shard1_replica1] o.a.s.s.IndexSchema [collection1_shard1_replica1] Schema name=minimal
[junit4] 2> 1337817 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 x:collection1_shard1_replica2] o.a.s.s.IndexSchema [collection1_shard1_replica2] Schema name=minimal
[junit4] 2> 1337836 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 x:collection1_shard1_replica2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1337836 INFO (qtp1790213708-10274) [n:127.0.0.1:62349_solr c:collection1 s:shard1 x:collection1_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1337839 INFO (qtp1790213708-10274) [n:127.0.0.1:62349_solr c:collection1 s:shard1 x:collection1_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica1' using configuration from collection collection1
[junit4] 2> 1337839 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 x:collection1_shard1_replica2] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica2' using configuration from collection collection1
[junit4] 2> 1337839 INFO (qtp1790213708-10274) [n:127.0.0.1:62349_solr c:collection1 s:shard1 x:collection1_shard1_replica1] o.a.s.c.SolrCore [[collection1_shard1_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReqParamsAPI_706A414369DC7549-001/tempDir-001/node1/collection1_shard1_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReqParamsAPI_706A414369DC7549-001/tempDir-001/node1/./collection1_shard1_replica1/data/]
[junit4] 2> 1337839 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 x:collection1_shard1_replica2] o.a.s.c.SolrCore [[collection1_shard1_replica2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReqParamsAPI_706A414369DC7549-001/tempDir-001/node2/collection1_shard1_replica2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReqParamsAPI_706A414369DC7549-001/tempDir-001/node2/./collection1_shard1_replica2/data/]
[junit4] 2> 1337923 INFO (qtp1790213708-10274) [n:127.0.0.1:62349_solr c:collection1 s:shard1 x:collection1_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1337923 INFO (qtp1790213708-10274) [n:127.0.0.1:62349_solr c:collection1 s:shard1 x:collection1_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1337925 INFO (qtp1790213708-10274) [n:127.0.0.1:62349_solr c:collection1 s:shard1 x:collection1_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1337925 INFO (qtp1790213708-10274) [n:127.0.0.1:62349_solr c:collection1 s:shard1 x:collection1_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1337926 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 x:collection1_shard1_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1337926 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 x:collection1_shard1_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1337928 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 x:collection1_shard1_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1337928 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 x:collection1_shard1_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1337929 INFO (qtp1790213708-10274) [n:127.0.0.1:62349_solr c:collection1 s:shard1 x:collection1_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2cbadf[collection1_shard1_replica1] main]
[junit4] 2> 1337931 INFO (qtp1790213708-10274) [n:127.0.0.1:62349_solr c:collection1 s:shard1 x:collection1_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1337932 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 x:collection1_shard1_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@630e48c2[collection1_shard1_replica2] main]
[junit4] 2> 1337932 INFO (qtp1790213708-10274) [n:127.0.0.1:62349_solr c:collection1 s:shard1 x:collection1_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1337932 INFO (qtp1790213708-10274) [n:127.0.0.1:62349_solr c:collection1 s:shard1 x:collection1_shard1_replica1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
[junit4] 2> 1337935 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 x:collection1_shard1_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1337936 INFO (qtp1790213708-10274) [n:127.0.0.1:62349_solr c:collection1 s:shard1 x:collection1_shard1_replica1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 1337936 INFO (qtp1790213708-10274) [n:127.0.0.1:62349_solr c:collection1 s:shard1 x:collection1_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1337936 INFO (qtp1790213708-10274) [n:127.0.0.1:62349_solr c:collection1 s:shard1 x:collection1_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559174088103034880
[junit4] 2> 1337938 INFO (searcherExecutor-4856-thread-1-processing-n:127.0.0.1:62349_solr x:collection1_shard1_replica1 s:shard1 c:collection1) [n:127.0.0.1:62349_solr c:collection1 s:shard1 x:collection1_shard1_replica1] o.a.s.c.SolrCore [collection1_shard1_replica1] Registered new searcher Searcher@2cbadf[collection1_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1337938 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 x:collection1_shard1_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1337939 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 x:collection1_shard1_replica2] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
[junit4] 2> 1337941 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 x:collection1_shard1_replica2] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 1337941 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 x:collection1_shard1_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1337942 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 x:collection1_shard1_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559174088109326336
[junit4] 2> 1337944 INFO (searcherExecutor-4857-thread-1-processing-n:127.0.0.1:62350_solr x:collection1_shard1_replica2 s:shard1 c:collection1) [n:127.0.0.1:62350_solr c:collection1 s:shard1 x:collection1_shard1_replica2] o.a.s.c.SolrCore [collection1_shard1_replica2] Registered new searcher Searcher@630e48c2[collection1_shard1_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1337956 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 x:collection1_shard1_replica2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1337956 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 x:collection1_shard1_replica2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1337957 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 x:collection1_shard1_replica2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:62350/solr/collection1_shard1_replica2/
[junit4] 2> 1337957 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 x:collection1_shard1_replica2] o.a.s.u.PeerSync PeerSync: core=collection1_shard1_replica2 url=http://127.0.0.1:62350/solr START replicas=[http://127.0.0.1:62349/solr/collection1_shard1_replica1/] nUpdates=100
[junit4] 2> 1337957 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 x:collection1_shard1_replica2] o.a.s.u.PeerSync PeerSync: core=collection1_shard1_replica2 url=http://127.0.0.1:62350/solr DONE. We have no versions. sync failed.
[junit4] 2> 1337959 INFO (qtp1790213708-10275) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.c.S.Request [collection1_shard1_replica1] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1337960 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 x:collection1_shard1_replica2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 1337960 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 x:collection1_shard1_replica2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 1337968 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 x:collection1_shard1_replica2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:62350/solr/collection1_shard1_replica2/ shard1
[junit4] 2> 1338074 INFO (zkCallback-1641-thread-1-processing-n:127.0.0.1:62350_solr) [n:127.0.0.1:62350_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1338074 INFO (zkCallback-1640-thread-2-processing-n:127.0.0.1:62349_solr) [n:127.0.0.1:62349_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1338129 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 x:collection1_shard1_replica2] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1338133 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 x:collection1_shard1_replica2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=collection1_shard1_replica2&action=CREATE&numShards=1&collection=collection1&shard=shard1&wt=javabin&version=2} status=0 QTime=1347
[junit4] 2> 1338241 INFO (zkCallback-1640-thread-1-processing-n:127.0.0.1:62349_solr) [n:127.0.0.1:62349_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1338242 INFO (zkCallback-1641-thread-1-processing-n:127.0.0.1:62350_solr) [n:127.0.0.1:62350_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1338962 INFO (qtp1790213708-10274) [n:127.0.0.1:62349_solr c:collection1 s:shard1 x:collection1_shard1_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=collection1_shard1_replica1&action=CREATE&numShards=1&collection=collection1&shard=shard1&wt=javabin&version=2} status=0 QTime=2181
[junit4] 2> 1338967 INFO (qtp1790213708-10273) [n:127.0.0.1:62349_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1339070 INFO (zkCallback-1641-thread-1-processing-n:127.0.0.1:62350_solr) [n:127.0.0.1:62350_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1339070 INFO (zkCallback-1640-thread-3-processing-n:127.0.0.1:62349_solr) [n:127.0.0.1:62349_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1339975 INFO (qtp1790213708-10273) [n:127.0.0.1:62349_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=conf1&name=collection1&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=3311
[junit4] 2> 1339995 INFO (TEST-TestReqParamsAPI.test-seed#[706A414369DC7549]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 1340002 INFO (TEST-TestReqParamsAPI.test-seed#[706A414369DC7549]) [ ] o.a.s.c.TestSolrConfigHandler going to send config command. path /config?wt=json , payload: {
[junit4] 2> 'create-requesthandler' : { 'name' : '/dump0', 'class': 'org.apache.solr.handler.DumpRequestHandler' }
[junit4] 2> }
[junit4] 2> 1340010 INFO (qtp1308645741-10284) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.SolrConfigHandler Executed config commands successfully and persisted to ZK [{"create-requesthandler":{
[junit4] 2> "name":"/dump0",
[junit4] 2> "class":"org.apache.solr.handler.DumpRequestHandler"}}]
[junit4] 2> 1340010 INFO (qtp1308645741-10284) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.SolrConfigHandler Waiting up to 30 secs for 2 replicas to set the property overlay to be of version 0 for collection collection1
[junit4] 2> 1340012 INFO (Thread-2358) [n:127.0.0.1:62350_solr ] o.a.s.c.SolrCore config update listener called for core collection1_shard1_replica2
[junit4] 2> 1340012 INFO (Thread-2357) [n:127.0.0.1:62349_solr ] o.a.s.c.SolrCore config update listener called for core collection1_shard1_replica1
[junit4] 2> 1340013 INFO (solrHandlerExecutor-4866-thread-1-processing-n:127.0.0.1:62350_solr x:collection1_shard1_replica2 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.SolrConfigHandler Time elapsed : 0 secs, maxWait 30
[junit4] 2> 1340013 INFO (solrHandlerExecutor-4866-thread-2-processing-n:127.0.0.1:62350_solr x:collection1_shard1_replica2 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.SolrConfigHandler Time elapsed : 0 secs, maxWait 30
[junit4] 2> 1340014 INFO (Thread-2358) [n:127.0.0.1:62350_solr ] o.a.s.c.SolrCore core reload collection1_shard1_replica2
[junit4] 2> 1340014 INFO (Thread-2357) [n:127.0.0.1:62349_solr ] o.a.s.c.SolrCore core reload collection1_shard1_replica1
[junit4] 2> 1340027 INFO (Thread-2357) [n:127.0.0.1:62349_solr ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1340027 INFO (Thread-2358) [n:127.0.0.1:62350_solr ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1340036 INFO (Thread-2358) [n:127.0.0.1:62350_solr ] o.a.s.s.IndexSchema [collection1_shard1_replica2] Schema name=minimal
[junit4] 2> 1340036 INFO (Thread-2357) [n:127.0.0.1:62349_solr ] o.a.s.s.IndexSchema [collection1_shard1_replica1] Schema name=minimal
[junit4] 2> 1340060 INFO (Thread-2357) [n:127.0.0.1:62349_solr ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1340060 INFO (Thread-2358) [n:127.0.0.1:62350_solr ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1340063 INFO (Thread-2358) [n:127.0.0.1:62350_solr ] o.a.s.c.CoreContainer Reloading SolrCore 'collection1_shard1_replica2' using configuration from collection collection1
[junit4] 2> 1340064 INFO (Thread-2357) [n:127.0.0.1:62349_solr ] o.a.s.c.CoreContainer Reloading SolrCore 'collection1_shard1_replica1' using configuration from collection collection1
[junit4] 2> 1340065 INFO (Thread-2358) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.c.SolrCore [[collection1_shard1_replica2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReqParamsAPI_706A414369DC7549-001/tempDir-001/node2/collection1_shard1_replica2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReqParamsAPI_706A414369DC7549-001/tempDir-001/node2/./collection1_shard1_replica2/data/]
[junit4] 2> 1340065 INFO (Thread-2357) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.c.SolrCore [[collection1_shard1_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReqParamsAPI_706A414369DC7549-001/tempDir-001/node1/collection1_shard1_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReqParamsAPI_706A414369DC7549-001/tempDir-001/node1/./collection1_shard1_replica1/data/]
[junit4] 2> 1340119 INFO (qtp1308645741-10285) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.SolrConfigHandler expecting overlay version 0 but my version is -1
[junit4] 2> 1340120 INFO (qtp1308645741-10285) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.c.S.Request [collection1_shard1_replica2] webapp=/solr path=/config/znodeVersion params={overlay=0&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1340120 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.SolrConfigHandler Trying to update my configs
[junit4] 2> 1340120 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.c.SolrCore config update listener called for core collection1_shard1_replica2
[junit4] 2> 1340122 INFO (solrHandlerExecutor-4866-thread-2-processing-n:127.0.0.1:62350_solr x:collection1_shard1_replica2 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.SolrConfigHandler Could not get expectedVersion 0 from http://127.0.0.1:62350/solr/collection1_shard1_replica2/ for prop overlay after 1 attempts
[junit4] 2> 1340122 INFO (solrHandlerExecutor-4866-thread-2-processing-n:127.0.0.1:62350_solr x:collection1_shard1_replica2 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.SolrConfigHandler Time elapsed : 0 secs, maxWait 30
[junit4] 2> 1340124 INFO (qtp1790213708-10276) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.h.SolrConfigHandler expecting overlay version 0 but my version is -1
[junit4] 2> 1340124 INFO (qtp1790213708-10276) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.c.S.Request [collection1_shard1_replica1] webapp=/solr path=/config/znodeVersion params={overlay=0&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1340124 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.h.SolrConfigHandler Trying to update my configs
[junit4] 2> 1340125 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.c.SolrCore config update listener called for core collection1_shard1_replica1
[junit4] 2> 1340125 INFO (solrHandlerExecutor-4866-thread-1-processing-n:127.0.0.1:62350_solr x:collection1_shard1_replica2 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.SolrConfigHandler Could not get expectedVersion 0 from http://127.0.0.1:62349/solr/collection1_shard1_replica1/ for prop overlay after 1 attempts
[junit4] 2> 1340125 INFO (solrHandlerExecutor-4866-thread-1-processing-n:127.0.0.1:62350_solr x:collection1_shard1_replica2 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.SolrConfigHandler Time elapsed : 0 secs, maxWait 30
[junit4] 2> 1340127 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.c.SolrCore core reload collection1_shard1_replica2
[junit4] 2> 1340130 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.c.SolrCore core reload collection1_shard1_replica1
[junit4] 2> 1340139 INFO (Thread-2357) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1340139 INFO (Thread-2357) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1340139 INFO (Thread-2357) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3231c347[collection1_shard1_replica1] main]
[junit4] 2> 1340139 INFO (Thread-2358) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1340139 INFO (Thread-2358) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1340139 INFO (Thread-2358) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@10078dd7[collection1_shard1_replica2] main]
[junit4] 2> 1340141 INFO (Thread-2357) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1340142 INFO (Thread-2358) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1340142 INFO (Thread-2357) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1340143 INFO (Thread-2357) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
[junit4] 2> 1340143 INFO (Thread-2358) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1340143 INFO (Thread-2358) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
[junit4] 2> 1340144 INFO (Thread-2357) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 1340144 INFO (Thread-2358) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 1340144 INFO (Thread-2357) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1340144 INFO (Thread-2358) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1340144 INFO (Thread-2358) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559174090418290688
[junit4] 2> 1340144 INFO (Thread-2357) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559174090418290688
[junit4] 2> 1340147 INFO (searcherExecutor-4867-thread-1-processing-n:127.0.0.1:62350_solr x:collection1_shard1_replica2 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.c.SolrCore [collection1_shard1_replica2] Registered new searcher Searcher@10078dd7[collection1_shard1_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1340147 INFO (searcherExecutor-4868-thread-1-processing-n:127.0.0.1:62349_solr x:collection1_shard1_replica1 s:shard1 c:collection1 r:core_node1) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.c.SolrCore [collection1_shard1_replica1] Registered new searcher Searcher@3231c347[collection1_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1340149 INFO (Thread-2357) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 1340149 INFO (Thread-2358) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 1340149 INFO (Thread-2357) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4d4b9f4[collection1_shard1_replica1] main]
[junit4] 2> 1340149 INFO (Thread-2358) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@105dc41e[collection1_shard1_replica2] main]
[junit4] 2> 1340149 INFO (Thread-2357) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.c.SolrCore [collection1_shard1_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@620c3b79
[junit4] 2> 1340149 INFO (searcherExecutor-4868-thread-1-processing-n:127.0.0.1:62349_solr x:collection1_shard1_replica1 s:shard1 c:collection1 r:core_node1) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.c.SolrCore [collection1_shard1_replica1] Registered new searcher Searcher@4d4b9f4[collection1_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1340149 INFO (searcherExecutor-4867-thread-1-processing-n:127.0.0.1:62350_solr x:collection1_shard1_replica2 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.c.SolrCore [collection1_shard1_replica2] Registered new searcher Searcher@105dc41e[collection1_shard1_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1340150 INFO (Thread-2357) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection1.shard1.replica1
[junit4] 2> 1340168 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1340168 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1340176 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.s.IndexSchema [collection1_shard1_replica1] Schema name=minimal
[junit4] 2> 1340176 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.s.IndexSchema [collection1_shard1_replica2] Schema name=minimal
[junit4] 2> 1340183 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1340212 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1340215 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.c.CoreContainer Reloading SolrCore 'collection1_shard1_replica2' using configuration from collection collection1
[junit4] 2> 1340215 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.c.CoreContainer Reloading SolrCore 'collection1_shard1_replica1' using configuration from collection collection1
[junit4] 2> 1340216 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.c.SolrCore [[collection1_shard1_replica2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReqParamsAPI_706A414369DC7549-001/tempDir-001/node2/collection1_shard1_replica2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReqParamsAPI_706A414369DC7549-001/tempDir-001/node2/./collection1_shard1_replica2/data/]
[junit4] 2> 1340216 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.c.SolrCore [[collection1_shard1_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReqParamsAPI_706A414369DC7549-001/tempDir-001/node1/collection1_shard1_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReqParamsAPI_706A414369DC7549-001/tempDir-001/node1/./collection1_shard1_replica1/data/]
[junit4] 2> 1340231 INFO (qtp1308645741-10286) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.SolrConfigHandler I already have the expected version 0 of config
[junit4] 2> 1340232 INFO (qtp1308645741-10286) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.SolrConfigHandler isStale false , resourceloader org.apache.solr.cloud.ZkSolrResourceLoader
[junit4] 2> 1340232 INFO (qtp1308645741-10286) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.c.S.Request [collection1_shard1_replica2] webapp=/solr path=/config/znodeVersion params={overlay=0&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1340238 INFO (qtp1790213708-10278) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.h.SolrConfigHandler I already have the expected version 0 of config
[junit4] 2> 1340238 INFO (qtp1790213708-10278) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.h.SolrConfigHandler isStale false , resourceloader org.apache.solr.cloud.ZkSolrResourceLoader
[junit4] 2> 1340238 INFO (qtp1790213708-10278) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.c.S.Request [collection1_shard1_replica1] webapp=/solr path=/config/znodeVersion params={overlay=0&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1340251 INFO (qtp1308645741-10284) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.SolrConfigHandler Took 241.0ms to set the property overlay to be of version 0 for collection collection1
[junit4] 2> 1340251 INFO (qtp1308645741-10284) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.c.S.Request [collection1_shard1_replica2] webapp=/solr path=/config params={wt=json} status=0 QTime=247
[junit4] 2> 1340251 INFO (qtp1308645741-10284) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.c.SolrCore [collection1_shard1_replica2] CLOSING SolrCore org.apache.solr.core.SolrCore@59455ab1
[junit4] 2> 1340253 INFO (qtp1308645741-10284) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection1.shard1.replica2
[junit4] 2> 1340256 INFO (TEST-TestReqParamsAPI.test-seed#[706A414369DC7549]) [ ] o.a.s.c.TestSolrConfigHandler going to send config command. path /config?wt=json , payload: {
[junit4] 2> 'create-requesthandler' : { 'name' : '/dump1', 'class': 'org.apache.solr.handler.DumpRequestHandler', 'useParams':'x' }
[junit4] 2> }
[junit4] 2> 1340290 INFO (qtp1308645741-10286) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.c.ZkController Persisted config data to node /configs/conf1/configoverlay.json
[junit4] 2> 1340295 INFO (qtp1308645741-10286) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.SolrConfigHandler Executed config commands successfully and persisted to ZK [{"create-requesthandler":{
[junit4] 2> "name":"/dump1",
[junit4] 2> "class":"org.apache.solr.handler.DumpRequestHandler",
[junit4] 2> "useParams":"x"}}]
[junit4] 2> 1340295 INFO (qtp1308645741-10286) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.SolrConfigHandler Waiting up to 30 secs for 2 replicas to set the property overlay to be of version 1 for collection collection1
[junit4] 2> 1340306 INFO (solrHandlerExecutor-4877-thread-2-processing-n:127.0.0.1:62350_solr x:collection1_shard1_replica2 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.SolrConfigHandler Time elapsed : 0 secs, maxWait 30
[junit4] 2> 1340306 INFO (Thread-2360) [n:127.0.0.1:62350_solr ] o.a.s.c.SolrCore config update listener called for core collection1_shard1_replica2
[junit4] 2> 1340307 INFO (Thread-2359) [n:127.0.0.1:62349_solr ] o.a.s.c.SolrCore config update listener called for core collection1_shard1_replica1
[junit4] 2> 1340307 INFO (solrHandlerExecutor-4877-thread-1-processing-n:127.0.0.1:62350_solr x:collection1_shard1_replica2 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.SolrConfigHandler Time elapsed : 0 secs, maxWait 30
[junit4] 2> 1340311 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1340312 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1340312 INFO (Thread-2360) [n:127.0.0.1:62350_solr ] o.a.s.c.SolrCore core reload collection1_shard1_replica2
[junit4] 2> 1340312 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1340312 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1340312 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7a3816e5[collection1_shard1_replica1] main]
[junit4] 2> 1340312 INFO (Thread-2359) [n:127.0.0.1:62349_solr ] o.a.s.c.SolrCore core reload collection1_shard1_replica1
[junit4] 2> 1340313 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@2c68ce31[collection1_shard1_replica2] main]
[junit4] 2> 1340314 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1340314 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1340315 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1340315 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
[junit4] 2> 1340315 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1340315 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
[junit4] 2> 1340317 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 1340317 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1340317 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559174090599694336
[junit4] 2> 1340319 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 1340319 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1340320 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559174090602840064
[junit4] 2> 1340322 INFO (searcherExecutor-4875-thread-1-processing-n:127.0.0.1:62349_solr x:collection1_shard1_replica1 s:shard1 c:collection1 r:core_node1) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.c.SolrCore [collection1_shard1_replica1] Registered new searcher Searcher@7a3816e5[collection1_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1340322 INFO (searcherExecutor-4876-thread-1-processing-n:127.0.0.1:62350_solr x:collection1_shard1_replica2 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.c.SolrCore [collection1_shard1_replica2] Registered new searcher Searcher@2c68ce31[collection1_shard1_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1340324 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 1340325 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 1340325 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@32d4f270[collection1_shard1_replica2] main]
[junit4] 2> 1340325 INFO (searcherExecutor-4876-thread-1-processing-n:127.0.0.1:62350_solr x:collection1_shard1_replica2 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.c.SolrCore [collection1_shard1_replica2] Registered new searcher Searcher@32d4f270[collection1_shard1_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1340333 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@766daf9c[collection1_shard1_replica1] main]
[junit4] 2> 1340333 INFO (searcherExecutor-4875-thread-1-processing-n:127.0.0.1:62349_solr x:collection1_shard1_replica1 s:shard1 c:collection1 r:core_node1) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.c.SolrCore [collection1_shard1_replica1] Registered new searcher Searcher@766daf9c[collection1_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1340334 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.c.SolrCore [collection1_shard1_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@397c5a99
[junit4] 2> 1340334 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection1.shard1.replica1
[junit4] 2> 1340342 INFO (Thread-2360) [n:127.0.0.1:62350_solr ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1340344 INFO (Thread-2359) [n:127.0.0.1:62349_solr ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1340350 INFO (Thread-2360) [n:127.0.0.1:62350_solr ] o.a.s.s.IndexSchema [collection1_shard1_replica2] Schema name=minimal
[junit4] 2> 1340352 INFO (Thread-2359) [n:127.0.0.1:62349_solr ] o.a.s.s.IndexSchema [collection1_shard1_replica1] Schema name=minimal
[junit4] 2> 1340356 INFO (Thread-2360) [n:127.0.0.1:62350_solr ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1340356 INFO (Thread-2359) [n:127.0.0.1:62349_solr ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1340384 INFO (Thread-2359) [n:127.0.0.1:62349_solr ] o.a.s.c.CoreContainer Reloading SolrCore 'collection1_shard1_replica1' using configuration from collection collection1
[junit4] 2> 1340384 INFO (Thread-2360) [n:127.0.0.1:62350_solr ] o.a.s.c.CoreContainer Reloading SolrCore 'collection1_shard1_replica2' using configuration from collection collection1
[junit4] 2> 1340385 INFO (Thread-2359) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.c.SolrCore [[collection1_shard1_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReqParamsAPI_706A414369DC7549-001/tempDir-001/node1/collection1_shard1_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReqParamsAPI_706A414369DC7549-001/tempDir-001/node1/./collection1_shard1_replica1/data/]
[junit4] 2> 1340385 INFO (Thread-2360) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.c.SolrCore [[collection1_shard1_replica2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReqParamsAPI_706A414369DC7549-001/tempDir-001/node2/collection1_shard1_replica2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReqParamsAPI_706A414369DC7549-001/tempDir-001/node2/./collection1_shard1_replica2/data/]
[junit4] 2> 1340410 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.SolrConfigHandler expecting overlay version 1 but my version is 0
[junit4] 2> 1340411 INFO (qtp1308645741-10281) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.c.S.Request [collection1_shard1_replica2] webapp=/solr path=/config/znodeVersion params={overlay=1&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 1340411 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.SolrConfigHandler Trying to update my configs
[junit4] 2> 1340411 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.c.SolrCore config update listener called for core collection1_shard1_replica2
[junit4] 2> 1340411 INFO (qtp1790213708-10278) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.h.SolrConfigHandler expecting overlay version 1 but my version is 0
[junit4] 2> 1340412 INFO (qtp1790213708-10278) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.c.S.Request [collection1_shard1_replica1] webapp=/solr path=/config/znodeVersion params={overlay=1&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1340413 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.h.SolrConfigHandler Trying to update my configs
[junit4] 2> 1340413 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.c.SolrCore config update listener called for core collection1_shard1_replica1
[junit4] 2> 1340414 INFO (solrHandlerExecutor-4877-thread-2-processing-n:127.0.0.1:62350_solr x:collection1_shard1_replica2 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.SolrConfigHandler Could not get expectedVersion 1 from http://127.0.0.1:62350/solr/collection1_shard1_replica2/ for prop overlay after 1 attempts
[junit4] 2> 1340414 INFO (solrHandlerExecutor-4877-thread-2-processing-n:127.0.0.1:62350_solr x:collection1_shard1_replica2 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.SolrConfigHandler Time elapsed : 0 secs, maxWait 30
[junit4] 2> 1340415 INFO (solrHandlerExecutor-4877-thread-1-processing-n:127.0.0.1:62350_solr x:collection1_shard1_replica2 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.SolrConfigHandler Could not get expectedVersion 1 from http://127.0.0.1:62349/solr/collection1_shard1_replica1/ for prop overlay after 1 attempts
[junit4] 2> 1340415 INFO (solrHandlerExecutor-4877-thread-1-processing-n:127.0.0.1:62350_solr x:collection1_shard1_replica2 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.SolrConfigHandler Time elapsed : 0 secs, maxWait 30
[junit4] 2> 1340416 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.c.SolrCore core reload collection1_shard1_replica2
[junit4] 2> 1340416 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.c.SolrCore core reload collection1_shard1_replica1
[junit4] 2> 1340497 INFO (Thread-2360) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1340497 INFO (Thread-2360) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1340498 INFO (Thread-2360) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@26dcdf54[collection1_shard1_replica2] main]
[junit4] 2> 1340498 INFO (Thread-2359) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1340498 INFO (Thread-2359) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1340499 INFO (Thread-2359) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2fe86ce1[collection1_shard1_replica1] main]
[junit4] 2> 1340500 INFO (Thread-2360) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1340500 INFO (Thread-2359) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1340500 INFO (Thread-2360) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1340500 INFO (Thread-2360) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
[junit4] 2> 1340501 INFO (Thread-2359) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1340501 INFO (Thread-2359) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
[junit4] 2> 1340502 INFO (Thread-2360) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 1340502 INFO (Thread-2360) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1340502 INFO (Thread-2359) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 1340502 INFO (searcherExecutor-4885-thread-1-processing-n:127.0.0.1:62350_solr x:collection1_shard1_replica2 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.c.SolrCore [collection1_shard1_replica2] Registered new searcher Searcher@26dcdf54[collection1_shard1_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1340502 INFO (Thread-2359) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1340502 INFO (Thread-2360) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559174090793680896
[junit4] 2> 1340502 INFO (Thread-2359) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559174090793680896
[junit4] 2> 1340502 INFO (searcherExecutor-4884-thread-1-processing-n:127.0.0.1:62349_solr x:collection1_shard1_replica1 s:shard1 c:collection1 r:core_node1) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.c.SolrCore [collection1_shard1_replica1] Registered new searcher Searcher@2fe86ce1[collection1_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1340505 INFO (Thread-2359) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 1340505 INFO (Thread-2360) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 1340505 INFO (Thread-2359) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@172a25ac[collection1_shard1_replica1] main]
[junit4] 2> 1340505 INFO (Thread-2360) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@2636e030[collection1_shard1_replica2] main]
[junit4] 2> 1340505 INFO (Thread-2359) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.c.SolrCore [collection1_shard1_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@a677143
[junit4] 2> 1340505 INFO (searcherExecutor-4884-thread-1-processing-n:127.0.0.1:62349_solr x:collection1_shard1_replica1 s:shard1 c:collection1 r:core_node1) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.c.SolrCore [collection1_shard1_replica1] Registered new searcher Searcher@172a25ac[collection1_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1340505 INFO (searcherExecutor-4885-thread-1-processing-n:127.0.0.1:62350_solr x:collection1_shard1_replica2 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.c.SolrCore [collection1_shard1_replica2] Registered new searcher Searcher@2636e030[collection1_shard1_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1340505 INFO (Thread-2359) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection1.shard1.replica1
[junit4] 2> 1340505 INFO (Thread-2360) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.c.SolrCore [collection1_shard1_replica2] CLOSING SolrCore org.apache.solr.core.SolrCore@322888a2
[junit4] 2> 1340506 INFO (Thread-2360) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection1.shard1.replica2
[junit4] 2> 1340518 INFO (qtp1308645741-10282) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.SolrConfigHandler I already have the expected version 1 of config
[junit4] 2> 1340518 INFO (qtp1308645741-10282) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.SolrConfigHandler isStale false , resourceloader org.apache.solr.cloud.ZkSolrResourceLoader
[junit4] 2> 1340518 INFO (qtp1308645741-10282) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.c.S.Request [collection1_shard1_replica2] webapp=/solr path=/config/znodeVersion params={overlay=1&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1340522 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1340522 INFO (SolrConfigHandler-refreshconf) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1340524 INFO (qtp1790213708-10275) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.h.SolrConfigHandler I already have the expected version 1 of config
[junit4] 2> 1340524 INFO (qtp1790213708-10275) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.h.SolrConfigHandler isStale false , resourceloader org.apache.solr.cloud.ZkSolrResourceLoader
[junit4] 2> 1340524 INFO (qtp1790213708-10275) [n:127.0.0.1:62349_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica1] o.a.s.c.S.Request [collection1_shard1_replica1] webapp=/solr path=/config/znodeVersion params={overlay=1&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1340526 INFO (qtp1308645741-10286) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.h.SolrConfigHandler Took 230.0ms to set the property overlay to be of version 1 for collection collection1
[junit4] 2> 1340526 INFO (qtp1308645741-10286) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.c.S.Request [collection1_shard1_replica2] webapp=/solr path=/config params={wt=json} status=0 QTime=268
[junit4] 2> 1340526 INFO (qtp1308645741-10286) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.c.SolrCore [collection1_shard1_replica2] CLOSING SolrCore org.apache.solr.core.SolrCore@531bf0f
[junit4] 2> 1340527 INFO (qtp1308645741-10286) [n:127.0.0.1:62350_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica2] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection1.shard1.replica2
[junit4] 2> 1340527 INFO (TEST-TestReqParamsAPI.test-seed#[706A414369DC7549]) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):
[...truncated too long message...]
=97448478748901394-127.0.0.1:65416_-n_0000000004) closing
[junit4] 2> 3048126 INFO (OverseerStateUpdate-97448478748901394-127.0.0.1:65416_-n_0000000004) [n:127.0.0.1:65416_ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:65416_
[junit4] 2> 3049639 WARN (zkCallback-3356-thread-3-processing-n:127.0.0.1:65416_) [n:127.0.0.1:65416_ ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 3049639 INFO (TEST-PeerSyncReplicationTest.test-seed#[706A414369DC7549]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 3049641 INFO (TEST-PeerSyncReplicationTest.test-seed#[706A414369DC7549]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@421758ed{HTTP/1.1,[http/1.1]}{127.0.0.1:65416}
[junit4] 2> 3049641 INFO (TEST-PeerSyncReplicationTest.test-seed#[706A414369DC7549]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6daea30{/,null,UNAVAILABLE}
[junit4] 2> 3049642 INFO (TEST-PeerSyncReplicationTest.test-seed#[706A414369DC7549]) [ ] o.a.s.c.ChaosMonkey monkey: stop shard! 65425
[junit4] 2> 3049645 INFO (TEST-PeerSyncReplicationTest.test-seed#[706A414369DC7549]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:65384 65384
[junit4] 2> 3049663 INFO (Thread-3183) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:65384 65384
[junit4] 2> 3053943 WARN (Thread-3183) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/aliases.json
[junit4] 2> 6 /solr/clusterprops.json
[junit4] 2> 5 /solr/security.json
[junit4] 2> 5 /solr/configs/conf1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/clusterstate.json
[junit4] 2> 4 /solr/collections/collection1/state.json
[junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/97448478748901385-core_node1-n_0000000000
[junit4] 2> 2 /solr/overseer_elect/election/97448478748901385-127.0.0.1:65412_-n_0000000001
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 207 /solr/overseer/collection-queue-work
[junit4] 2> 44 /solr/overseer/queue
[junit4] 2> 16 /solr/overseer/queue-work
[junit4] 2> 6 /solr/collections
[junit4] 2> 5 /solr/live_nodes
[junit4] 2>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=PeerSyncReplicationTest -Dtests.method=test -Dtests.seed=706A414369DC7549 -Dtests.slow=true -Dtests.locale=es-ES -Dtests.timezone=Navajo -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 217s J0 | PeerSyncReplicationTest.test <<<
[junit4] > Throwable #1: java.lang.AssertionError: timeout waiting to see all nodes active
[junit4] > at __randomizedtesting.SeedInfo.seed([706A414369DC7549:F83E7E99C72018B1]:0)
[junit4] > at org.apache.solr.cloud.PeerSyncReplicationTest.waitTillNodesActive(PeerSyncReplicationTest.java:326)
[junit4] > at org.apache.solr.cloud.PeerSyncReplicationTest.bringUpDeadNodeAndEnsureNoReplication(PeerSyncReplicationTest.java:277)
[junit4] > at org.apache.solr.cloud.PeerSyncReplicationTest.forceNodeFailureAndDoPeerSync(PeerSyncReplicationTest.java:259)
[junit4] > at org.apache.solr.cloud.PeerSyncReplicationTest.test(PeerSyncReplicationTest.java:138)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 3053955 INFO (SUITE-PeerSyncReplicationTest-seed#[706A414369DC7549]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_706A414369DC7549-001
[junit4] 2> NOTE: test params are: codec=Lucene70, sim=RandomSimilarity(queryNorm=false): {}, locale=es-ES, timezone=Navajo
[junit4] 2> NOTE: Mac OS X 10.11.6 x86_64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=3,threads=1,free=154011680,total=518979584
[junit4] 2> NOTE: All tests run in this JVM: [BinaryUpdateRequestHandlerTest, UtilsToolTest, TestUpdate, MultiTermTest, TestSegmentSorting, TestPerFieldSimilarity, TestCollationField, TestSolrDynamicMBean, SyncSliceTest, StressHdfsTest, TestMinMaxOnMultiValuedField, TestFastOutputStream, CdcrVersionReplicationTest, ConfigSetsAPITest, TestDelegationWithHadoopAuth, IndexBasedSpellCheckerTest, TestDFISimilarityFactory, HardAutoCommitTest, TestSubQueryTransformerCrossCore, SuggestComponentTest, DocumentAnalysisRequestHandlerTest, DOMUtilTest, TestSchemalessBufferedUpdates, TestPseudoReturnFields, TestOverriddenPrefixQueryForCustomFieldType, TestJmxIntegration, TestSolrCLIRunExample, TestSizeLimitedDistributedMap, TestCSVResponseWriter, SolrIndexConfigTest, CurrencyFieldXmlFileTest, ClassificationUpdateProcessorIntegrationTest, DirectSolrConnectionTest, TestRuleBasedAuthorizationPlugin, TestFieldCacheVsDocValues, DistributedFacetPivotWhiteBoxTest, HighlighterTest, SpatialFilterTest, LeaderElectionContextKeyTest, TestRawTransformer, TestLegacyNumericRangeQueryBuilder, TestIBSimilarityFactory, TestManagedSynonymFilterFactory, TestFileDictionaryLookup, RequestLoggingTest, PathHierarchyTokenizerFactoryTest, TestTolerantUpdateProcessorRandomCloud, TestBinaryField, TestSort, TestSolr4Spatial2, TestXIncludeConfig, HdfsThreadLeakTest, RulesTest, TestExtendedDismaxParser, AliasIntegrationTest, DirectSolrSpellCheckerTest, CloneFieldUpdateProcessorFactoryTest, TestRestoreCore, TestRandomDVFaceting, VersionInfoTest, ResponseHeaderTest, NumericFieldsTest, TestDistribDocBasedVersion, OverseerTaskQueueTest, TestExactSharedStatsCache, TestSolrDeletionPolicy1, EnumFieldTest, CdcrReplicationHandlerTest, StatelessScriptUpdateProcessorFactoryTest, OverseerCollectionConfigSetProcessorTest, TestFilteredDocIdSet, CursorPagingTest, TestReplicaProperties, CoreSorterTest, FacetPivotSmallTest, TestPostingsSolrHighlighter, TestCollationFieldDocValues, TestStressUserVersions, TestNRTOpen, TestSQLHandlerNonCloud, SolrCoreTest, TestBinaryResponseWriter, DistributedVersionInfoTest, ForceLeaderTest, TestDefaultSearchFieldResource, TestBM25SimilarityFactory, DocValuesMultiTest, TestSolrCloudWithKerberosAlt, DeleteNodeTest, AtomicUpdatesTest, TestReRankQParserPlugin, TestOnReconnectListenerSupport, RecoveryAfterSoftCommitTest, SolrCLIZkUtilsTest, TestDefaultStatsCache, BlockJoinFacetDistribTest, TolerantUpdateProcessorTest, ReplaceNodeTest, OutputWriterTest, TestMacroExpander, TestSlowCompositeReaderWrapper, ConnectionReuseTest, TestTrackingShardHandlerFactory, TestHdfsUpdateLog, TestMissingGroups, TestCSVLoader, RequiredFieldsTest, SimplePostToolTest, UniqFieldsUpdateProcessorFactoryTest, HighlighterMaxOffsetTest, ExitableDirectoryReaderTest, TestStressLiveNodes, TestJsonFacetRefinement, TestInfoStreamLogging, TestNumericTerms64, TestDocTermOrds, TestQuerySenderNoQuery, SuggesterFSTTest, SoftAutoCommitTest, BasicDistributedZkTest, DistributedExpandComponentTest, OpenExchangeRatesOrgProviderTest, CreateCollectionCleanupTest, ZkNodePropsTest, TestCollectionAPI, ZkControllerTest, TestEmbeddedSolrServerConstructors, RecoveryZkTest, PrimitiveFieldTypeTest, JvmMetricsTest, ShardSplitTest, HdfsBasicDistributedZk2Test, TestSolrQueryParserDefaultOperatorResource, TestDocBasedVersionConstraints, TestHighFrequencyDictionaryFactory, TestDocumentBuilder, CheckHdfsIndexTest, TemplateUpdateProcessorTest, SampleTest, UnloadDistributedZkTest, MissingSegmentRecoveryTest, TermsComponentTest, MoreLikeThisHandlerTest, SearchHandlerTest, BooleanFieldTest, TestJsonRequest, TestPerFieldSimilarityWithDefaultOverride, TestSurroundQueryParser, SolrRequestParserTest, TestFaceting, DocumentBuilderTest, TestSortByMinMaxFunction, HdfsRecoverLeaseTest, TestStressLucene, TestExpandComponent, SimpleFacetsTest, TestStressVersions, TestCloudRecovery, ShardRoutingTest, StatsReloadRaceTest, TestCloudDeleteByQuery, PluginInfoTest, HdfsCollectionsAPIDistributedZkTest, TestSimpleQParserPlugin, TestLRUCache, LeaderElectionTest, TestZkChroot, ShardRoutingCustomTest, TestDistributedSearch, TestHashPartitioner, TermVectorComponentDistributedTest, TestRangeQuery, SuggesterTSTTest, TestTrie, PolyFieldTest, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, SolrCoreCheckLockOnStartupTest, FieldMutatingUpdateProcessorTest, TestAtomicUpdateErrorCases, TestWordDelimiterFilterFactory, QueryEqualityTest, DefaultValueUpdateProcessorTest, SortByFunctionTest, DistanceFunctionTest, DisMaxRequestHandlerTest, StandardRequestHandlerTest, TestWriterPerf, RequestHandlersTest, TestIndexingPerformance, TestSolrQueryParser, IndexSchemaRuntimeFieldTest, RegexBoostProcessorTest, IndexSchemaTest, QueryParsingTest, TestPartialUpdateDeduplication, PingRequestHandlerTest, HighlighterConfigTest, SOLR749Test, AlternateDirectoryTest, CopyFieldTest, ResponseLogComponentTest, TestStressRecovery, ResourceLoaderTest, TestFastLRUCache, DateMathParserTest, ChaosMonkeyNothingIsSafeTest, TestTolerantSearch, ActionThrottleTest, CdcrBootstrapTest, CollectionsAPIAsyncDistributedZkTest, CollectionsAPISolrJTest, DeleteLastCustomShardedReplicaTest, DistribCursorPagingTest, LeaderFailoverAfterPartitionTest, LeaderFailureAfterFreshStartTest, LeaderInitiatedRecoveryOnShardRestartTest, OutOfBoxZkACLAndCredentialsProvidersTest, OverseerRolesTest, PeerSyncReplicationTest]
[junit4] Completed [518/694 (4!)] on J0 in 216.89s, 1 test, 1 failure <<< FAILURES!
[...truncated 63112 lines...]
[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3827
- Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3827/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC
1 tests failed.
FAILED: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Expected to see collection awhollynewcollection_0 null Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/8)={ "replicationFactor":"3", "shards":{ "shard1":{ "range":"80000000-b332ffff", "state":"active", "replicas":{ "core_node4":{ "core":"awhollynewcollection_0_shard1_replica1", "base_url":"http://127.0.0.1:56265/solr", "node_name":"127.0.0.1:56265_solr", "state":"active"}, "core_node5":{ "core":"awhollynewcollection_0_shard1_replica3", "base_url":"http://127.0.0.1:56266/solr", "node_name":"127.0.0.1:56266_solr", "state":"active", "leader":"true"}}}, "shard2":{ "range":"b3330000-e665ffff", "state":"active", "replicas":{}}, "shard3":{ "range":"e6660000-1998ffff", "state":"active", "replicas":{ "core_node1":{ "core":"awhollynewcollection_0_shard3_replica3", "base_url":"http://127.0.0.1:56265/solr", "node_name":"127.0.0.1:56265_solr", "state":"active"}, "core_node6":{ "core":"awhollynewcollection_0_shard3_replica1", "base_url":"http://127.0.0.1:56266/solr", "node_name":"127.0.0.1:56266_solr", "state":"active", "leader":"true"}, "core_node9":{ "core":"awhollynewcollection_0_shard3_replica2", "base_url":"http://127.0.0.1:56264/solr", "node_name":"127.0.0.1:56264_solr", "state":"active"}}}, "shard4":{ "range":"19990000-4ccbffff", "state":"active", "replicas":{ "core_node2":{ "core":"awhollynewcollection_0_shard4_replica3", "base_url":"http://127.0.0.1:56264/solr", "node_name":"127.0.0.1:56264_solr", "state":"active", "leader":"true"}, "core_node3":{ "core":"awhollynewcollection_0_shard4_replica2", "base_url":"http://127.0.0.1:56266/solr", "node_name":"127.0.0.1:56266_solr", "state":"active"}}}, "shard5":{ "range":"4ccc0000-7fffffff", "state":"active", "replicas":{"core_node7":{ "core":"awhollynewcollection_0_shard5_replica3", "base_url":"http://127.0.0.1:56266/solr", "node_name":"127.0.0.1:56266_solr", "state":"active", "leader":"true"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"4", "autoAddReplicas":"false"}
Stack Trace:
java.lang.AssertionError: Expected to see collection awhollynewcollection_0
null
Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/8)={
"replicationFactor":"3",
"shards":{
"shard1":{
"range":"80000000-b332ffff",
"state":"active",
"replicas":{
"core_node4":{
"core":"awhollynewcollection_0_shard1_replica1",
"base_url":"http://127.0.0.1:56265/solr",
"node_name":"127.0.0.1:56265_solr",
"state":"active"},
"core_node5":{
"core":"awhollynewcollection_0_shard1_replica3",
"base_url":"http://127.0.0.1:56266/solr",
"node_name":"127.0.0.1:56266_solr",
"state":"active",
"leader":"true"}}},
"shard2":{
"range":"b3330000-e665ffff",
"state":"active",
"replicas":{}},
"shard3":{
"range":"e6660000-1998ffff",
"state":"active",
"replicas":{
"core_node1":{
"core":"awhollynewcollection_0_shard3_replica3",
"base_url":"http://127.0.0.1:56265/solr",
"node_name":"127.0.0.1:56265_solr",
"state":"active"},
"core_node6":{
"core":"awhollynewcollection_0_shard3_replica1",
"base_url":"http://127.0.0.1:56266/solr",
"node_name":"127.0.0.1:56266_solr",
"state":"active",
"leader":"true"},
"core_node9":{
"core":"awhollynewcollection_0_shard3_replica2",
"base_url":"http://127.0.0.1:56264/solr",
"node_name":"127.0.0.1:56264_solr",
"state":"active"}}},
"shard4":{
"range":"19990000-4ccbffff",
"state":"active",
"replicas":{
"core_node2":{
"core":"awhollynewcollection_0_shard4_replica3",
"base_url":"http://127.0.0.1:56264/solr",
"node_name":"127.0.0.1:56264_solr",
"state":"active",
"leader":"true"},
"core_node3":{
"core":"awhollynewcollection_0_shard4_replica2",
"base_url":"http://127.0.0.1:56266/solr",
"node_name":"127.0.0.1:56266_solr",
"state":"active"}}},
"shard5":{
"range":"4ccc0000-7fffffff",
"state":"active",
"replicas":{"core_node7":{
"core":"awhollynewcollection_0_shard5_replica3",
"base_url":"http://127.0.0.1:56266/solr",
"node_name":"127.0.0.1:56266_solr",
"state":"active",
"leader":"true"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"4",
"autoAddReplicas":"false"}
at __randomizedtesting.SeedInfo.seed([31C36F6FB1443BEF:79B61BDBB777147A]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:265)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:503)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11906 lines...]
[junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4] 2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_31C36F6FB1443BEF-001/init-core-data-001
[junit4] 2> 1998145 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[31C36F6FB1443BEF]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 1998150 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[31C36F6FB1443BEF]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN) w/ MAC_OS_X supressed clientAuth
[junit4] 2> 1998151 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[31C36F6FB1443BEF]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_31C36F6FB1443BEF-001/tempDir-001
[junit4] 2> 1998152 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[31C36F6FB1443BEF]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1998152 INFO (Thread-2725) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1998152 INFO (Thread-2725) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1998256 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[31C36F6FB1443BEF]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:56261
[junit4] 2> 1998276 INFO (jetty-launcher-1941-thread-1) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1998276 INFO (jetty-launcher-1941-thread-2) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1998276 INFO (jetty-launcher-1941-thread-3) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1998277 INFO (jetty-launcher-1941-thread-4) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1998284 INFO (jetty-launcher-1941-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@22099fb3{/solr,null,AVAILABLE}
[junit4] 2> 1998285 INFO (jetty-launcher-1941-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7e66d712{/solr,null,AVAILABLE}
[junit4] 2> 1998285 INFO (jetty-launcher-1941-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3e6669f{/solr,null,AVAILABLE}
[junit4] 2> 1998286 INFO (jetty-launcher-1941-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5723c47a{/solr,null,AVAILABLE}
[junit4] 2> 1998291 INFO (jetty-launcher-1941-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@74ffa3d9{HTTP/1.1,[http/1.1]}{127.0.0.1:56263}
[junit4] 2> 1998291 INFO (jetty-launcher-1941-thread-1) [ ] o.e.j.s.Server Started @2003178ms
[junit4] 2> 1998291 INFO (jetty-launcher-1941-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=56263}
[junit4] 2> 1998291 ERROR (jetty-launcher-1941-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1998291 INFO (jetty-launcher-1941-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@58f726ad{HTTP/1.1,[http/1.1]}{127.0.0.1:56265}
[junit4] 2> 1998291 INFO (jetty-launcher-1941-thread-4) [ ] o.e.j.s.Server Started @2003179ms
[junit4] 2> 1998292 INFO (jetty-launcher-1941-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=56265}
[junit4] 2> 1998292 ERROR (jetty-launcher-1941-thread-4) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1998293 INFO (jetty-launcher-1941-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 1998293 INFO (jetty-launcher-1941-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 1998293 INFO (jetty-launcher-1941-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1998293 INFO (jetty-launcher-1941-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1998293 INFO (jetty-launcher-1941-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1998293 INFO (jetty-launcher-1941-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1998293 INFO (jetty-launcher-1941-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-12T12:20:48.159Z
[junit4] 2> 1998293 INFO (jetty-launcher-1941-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-12T12:20:48.159Z
[junit4] 2> 1998293 INFO (jetty-launcher-1941-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4f9edf3d{HTTP/1.1,[http/1.1]}{127.0.0.1:56264}
[junit4] 2> 1998293 INFO (jetty-launcher-1941-thread-3) [ ] o.e.j.s.Server Started @2003180ms
[junit4] 2> 1998293 INFO (jetty-launcher-1941-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=56264}
[junit4] 2> 1998293 ERROR (jetty-launcher-1941-thread-3) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1998298 INFO (jetty-launcher-1941-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 1998298 INFO (jetty-launcher-1941-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1998298 INFO (jetty-launcher-1941-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1998298 INFO (jetty-launcher-1941-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-12T12:20:48.164Z
[junit4] 2> 1998299 INFO (jetty-launcher-1941-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3668f136{HTTP/1.1,[http/1.1]}{127.0.0.1:56266}
[junit4] 2> 1998299 INFO (jetty-launcher-1941-thread-2) [ ] o.e.j.s.Server Started @2003186ms
[junit4] 2> 1998299 INFO (jetty-launcher-1941-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=56266}
[junit4] 2> 1998299 ERROR (jetty-launcher-1941-thread-2) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1998300 INFO (jetty-launcher-1941-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 1998300 INFO (jetty-launcher-1941-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1998300 INFO (jetty-launcher-1941-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1998300 INFO (jetty-launcher-1941-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-12T12:20:48.166Z
[junit4] 2> 1998303 INFO (jetty-launcher-1941-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1998304 INFO (jetty-launcher-1941-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1998305 INFO (jetty-launcher-1941-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1998305 INFO (jetty-launcher-1941-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1998338 INFO (jetty-launcher-1941-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56261/solr
[junit4] 2> 1998338 INFO (jetty-launcher-1941-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56261/solr
[junit4] 2> 1998338 INFO (jetty-launcher-1941-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56261/solr
[junit4] 2> 1998339 INFO (jetty-launcher-1941-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56261/solr
[junit4] 2> 1998407 INFO (jetty-launcher-1941-thread-1) [n:127.0.0.1:56263_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:56263_solr
[junit4] 2> 1998409 INFO (jetty-launcher-1941-thread-3) [n:127.0.0.1:56264_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56264_solr
[junit4] 2> 1998409 INFO (jetty-launcher-1941-thread-4) [n:127.0.0.1:56265_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56265_solr
[junit4] 2> 1998409 INFO (jetty-launcher-1941-thread-2) [n:127.0.0.1:56266_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56266_solr
[junit4] 2> 1998410 INFO (jetty-launcher-1941-thread-1) [n:127.0.0.1:56263_solr ] o.a.s.c.Overseer Overseer (id=97445612619104267-127.0.0.1:56263_solr-n_0000000000) starting
[junit4] 2> 1998414 INFO (zkCallback-1959-thread-1-processing-n:127.0.0.1:56266_solr) [n:127.0.0.1:56266_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1998414 INFO (zkCallback-1960-thread-1-processing-n:127.0.0.1:56264_solr) [n:127.0.0.1:56264_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1998415 INFO (zkCallback-1961-thread-1-processing-n:127.0.0.1:56265_solr) [n:127.0.0.1:56265_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1998415 INFO (zkCallback-1962-thread-1-processing-n:127.0.0.1:56263_solr) [n:127.0.0.1:56263_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1998440 INFO (jetty-launcher-1941-thread-1) [n:127.0.0.1:56263_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56263_solr
[junit4] 2> 1998442 INFO (zkCallback-1962-thread-1-processing-n:127.0.0.1:56263_solr) [n:127.0.0.1:56263_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1998442 INFO (zkCallback-1959-thread-1-processing-n:127.0.0.1:56266_solr) [n:127.0.0.1:56266_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1998443 INFO (zkCallback-1961-thread-1-processing-n:127.0.0.1:56265_solr) [n:127.0.0.1:56265_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1998443 INFO (zkCallback-1960-thread-1-processing-n:127.0.0.1:56264_solr) [n:127.0.0.1:56264_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1998564 INFO (jetty-launcher-1941-thread-2) [n:127.0.0.1:56266_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_31C36F6FB1443BEF-001/tempDir-001/node2/.
[junit4] 2> 1998568 INFO (jetty-launcher-1941-thread-3) [n:127.0.0.1:56264_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_31C36F6FB1443BEF-001/tempDir-001/node3/.
[junit4] 2> 1998568 INFO (jetty-launcher-1941-thread-4) [n:127.0.0.1:56265_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_31C36F6FB1443BEF-001/tempDir-001/node4/.
[junit4] 2> 1998570 INFO (jetty-launcher-1941-thread-1) [n:127.0.0.1:56263_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_31C36F6FB1443BEF-001/tempDir-001/node1/.
[junit4] 2> 2000571 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[31C36F6FB1443BEF]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2000573 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[31C36F6FB1443BEF]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:56261/solr ready
[junit4] 2> 2000651 INFO (TEST-CollectionsAPIDistributedZkTest.testCreationAndDeletion-seed#[31C36F6FB1443BEF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCreationAndDeletion
[junit4] 2> 2000653 INFO (TEST-CollectionsAPIDistributedZkTest.testCreationAndDeletion-seed#[31C36F6FB1443BEF]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2000659 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=created_and_deleted&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2000663 INFO (OverseerThreadFactory-6020-thread-1-processing-n:127.0.0.1:56263_solr) [n:127.0.0.1:56263_solr ] o.a.s.c.CreateCollectionCmd Create collection created_and_deleted
[junit4] 2> 2000782 INFO (qtp1715910283-12715) [n:127.0.0.1:56263_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=created_and_deleted_shard1_replica1&action=CREATE&numShards=1&collection=created_and_deleted&shard=shard1&wt=javabin&version=2
[junit4] 2> 2000898 INFO (zkCallback-1962-thread-1-processing-n:127.0.0.1:56263_solr) [n:127.0.0.1:56263_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/created_and_deleted/state.json] for collection [created_and_deleted] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2001805 INFO (qtp1715910283-12715) [n:127.0.0.1:56263_solr c:created_and_deleted s:shard1 x:created_and_deleted_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2001812 INFO (qtp1715910283-12715) [n:127.0.0.1:56263_solr c:created_and_deleted s:shard1 x:created_and_deleted_shard1_replica1] o.a.s.s.IndexSchema [created_and_deleted_shard1_replica1] Schema name=minimal
[junit4] 2> 2001817 INFO (qtp1715910283-12715) [n:127.0.0.1:56263_solr c:created_and_deleted s:shard1 x:created_and_deleted_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2001819 INFO (qtp1715910283-12715) [n:127.0.0.1:56263_solr c:created_and_deleted s:shard1 x:created_and_deleted_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'created_and_deleted_shard1_replica1' using configuration from collection created_and_deleted
[junit4] 2> 2001819 INFO (qtp1715910283-12715) [n:127.0.0.1:56263_solr c:created_and_deleted s:shard1 x:created_and_deleted_shard1_replica1] o.a.s.c.SolrCore [[created_and_deleted_shard1_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_31C36F6FB1443BEF-001/tempDir-001/node1/created_and_deleted_shard1_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_31C36F6FB1443BEF-001/tempDir-001/node1/./created_and_deleted_shard1_replica1/data/]
[junit4] 2> 2001903 INFO (qtp1715910283-12715) [n:127.0.0.1:56263_solr c:created_and_deleted s:shard1 x:created_and_deleted_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2001903 INFO (qtp1715910283-12715) [n:127.0.0.1:56263_solr c:created_and_deleted s:shard1 x:created_and_deleted_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2001906 INFO (qtp1715910283-12715) [n:127.0.0.1:56263_solr c:created_and_deleted s:shard1 x:created_and_deleted_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2001906 INFO (qtp1715910283-12715) [n:127.0.0.1:56263_solr c:created_and_deleted s:shard1 x:created_and_deleted_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2001907 INFO (qtp1715910283-12715) [n:127.0.0.1:56263_solr c:created_and_deleted s:shard1 x:created_and_deleted_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@61163de8[created_and_deleted_shard1_replica1] main]
[junit4] 2> 2001910 INFO (qtp1715910283-12715) [n:127.0.0.1:56263_solr c:created_and_deleted s:shard1 x:created_and_deleted_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2001910 INFO (qtp1715910283-12715) [n:127.0.0.1:56263_solr c:created_and_deleted s:shard1 x:created_and_deleted_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2001910 INFO (qtp1715910283-12715) [n:127.0.0.1:56263_solr c:created_and_deleted s:shard1 x:created_and_deleted_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2001911 INFO (searcherExecutor-6029-thread-1-processing-n:127.0.0.1:56263_solr x:created_and_deleted_shard1_replica1 s:shard1 c:created_and_deleted) [n:127.0.0.1:56263_solr c:created_and_deleted s:shard1 x:created_and_deleted_shard1_replica1] o.a.s.c.SolrCore [created_and_deleted_shard1_replica1] Registered new searcher Searcher@61163de8[created_and_deleted_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2001911 INFO (qtp1715910283-12715) [n:127.0.0.1:56263_solr c:created_and_deleted s:shard1 x:created_and_deleted_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559129805844119552
[junit4] 2> 2001921 INFO (qtp1715910283-12715) [n:127.0.0.1:56263_solr c:created_and_deleted s:shard1 x:created_and_deleted_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2001921 INFO (qtp1715910283-12715) [n:127.0.0.1:56263_solr c:created_and_deleted s:shard1 x:created_and_deleted_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2001921 INFO (qtp1715910283-12715) [n:127.0.0.1:56263_solr c:created_and_deleted s:shard1 x:created_and_deleted_shard1_replica1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:56263/solr/created_and_deleted_shard1_replica1/
[junit4] 2> 2001921 INFO (qtp1715910283-12715) [n:127.0.0.1:56263_solr c:created_and_deleted s:shard1 x:created_and_deleted_shard1_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2001921 INFO (qtp1715910283-12715) [n:127.0.0.1:56263_solr c:created_and_deleted s:shard1 x:created_and_deleted_shard1_replica1] o.a.s.c.SyncStrategy http://127.0.0.1:56263/solr/created_and_deleted_shard1_replica1/ has no replicas
[junit4] 2> 2001927 INFO (qtp1715910283-12715) [n:127.0.0.1:56263_solr c:created_and_deleted s:shard1 x:created_and_deleted_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:56263/solr/created_and_deleted_shard1_replica1/ shard1
[junit4] 2> 2002039 INFO (zkCallback-1962-thread-1-processing-n:127.0.0.1:56263_solr) [n:127.0.0.1:56263_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/created_and_deleted/state.json] for collection [created_and_deleted] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2002088 INFO (qtp1715910283-12715) [n:127.0.0.1:56263_solr c:created_and_deleted s:shard1 x:created_and_deleted_shard1_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2002094 INFO (qtp1715910283-12715) [n:127.0.0.1:56263_solr c:created_and_deleted s:shard1 x:created_and_deleted_shard1_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=created_and_deleted_shard1_replica1&action=CREATE&numShards=1&collection=created_and_deleted&shard=shard1&wt=javabin&version=2} status=0 QTime=1313
[junit4] 2> 2002102 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 2002202 INFO (zkCallback-1962-thread-2-processing-n:127.0.0.1:56263_solr) [n:127.0.0.1:56263_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/created_and_deleted/state.json] for collection [created_and_deleted] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2002675 INFO (OverseerCollectionConfigSetProcessor-97445612619104267-127.0.0.1:56263_solr-n_0000000000) [n:127.0.0.1:56263_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2003106 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&name=created_and_deleted&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=2447
[junit4] 2> 2003107 INFO (qtp1067139960-12738) [n:127.0.0.1:56266_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2003108 INFO (qtp1067139960-12738) [n:127.0.0.1:56266_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2003110 INFO (qtp1067139960-12739) [n:127.0.0.1:56266_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=created_and_deleted&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2003113 INFO (OverseerThreadFactory-6020-thread-2-processing-n:127.0.0.1:56263_solr) [n:127.0.0.1:56263_solr ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
[junit4] 2> 2003114 INFO (qtp1715910283-12716) [n:127.0.0.1:56263_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.created_and_deleted.shard1.replica1
[junit4] 2> 2003114 INFO (qtp1715910283-12716) [n:127.0.0.1:56263_solr ] o.a.s.c.SolrCore [created_and_deleted_shard1_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@64232be1
[junit4] 2> 2003115 INFO (qtp1715910283-12716) [n:127.0.0.1:56263_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.created_and_deleted.shard1.replica1
[junit4] 2> 2003120 INFO (qtp1715910283-12716) [n:127.0.0.1:56263_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=created_and_deleted_shard1_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=6
[junit4] 2> 2003842 INFO (qtp1067139960-12739) [n:127.0.0.1:56266_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=created_and_deleted&action=DELETE&wt=javabin&version=2} status=0 QTime=733
[junit4] 2> 2003843 INFO (qtp1067139960-12740) [n:127.0.0.1:56266_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2003843 INFO (qtp1067139960-12740) [n:127.0.0.1:56266_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2003844 INFO (TEST-CollectionsAPIDistributedZkTest.testCreationAndDeletion-seed#[31C36F6FB1443BEF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testCreationAndDeletion
[junit4] 2> 2003877 INFO (TEST-CollectionsAPIDistributedZkTest.testDeleteNonExistentCollection-seed#[31C36F6FB1443BEF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testDeleteNonExistentCollection
[junit4] 2> 2003880 INFO (TEST-CollectionsAPIDistributedZkTest.testDeleteNonExistentCollection-seed#[31C36F6FB1443BEF]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2003883 INFO (qtp1067139960-12741) [n:127.0.0.1:56266_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=unknown_collection&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2003888 INFO (OverseerCollectionConfigSetProcessor-97445612619104267-127.0.0.1:56263_solr-n_0000000000) [n:127.0.0.1:56263_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2003888 INFO (OverseerThreadFactory-6020-thread-3-processing-n:127.0.0.1:56263_solr) [n:127.0.0.1:56263_solr ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
[junit4] 2> 2003889 ERROR (OverseerThreadFactory-6020-thread-3-processing-n:127.0.0.1:56263_solr) [n:127.0.0.1:56263_solr ] o.a.s.c.OverseerCollectionMessageHandler Collection: unknown_collection operation: delete failed:org.apache.solr.common.SolrException: Could not find collection : unknown_collection
[junit4] 2> at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:194)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionMessageHandler.collectionCmd(OverseerCollectionMessageHandler.java:843)
[junit4] 2> at org.apache.solr.cloud.DeleteCollectionCmd.call(DeleteCollectionCmd.java:88)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:224)
[junit4] 2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:463)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 2003901 INFO (qtp1067139960-12741) [n:127.0.0.1:56266_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=unknown_collection&action=DELETE&wt=javabin&version=2} status=400 QTime=17
[junit4] 2> 2003902 INFO (qtp1067139960-12742) [n:127.0.0.1:56266_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&name=acollectionafterbaddelete&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2003905 INFO (OverseerThreadFactory-6020-thread-4-processing-n:127.0.0.1:56263_solr) [n:127.0.0.1:56263_solr ] o.a.s.c.CreateCollectionCmd Create collection acollectionafterbaddelete
[junit4] 2> 2003906 INFO (OverseerCollectionConfigSetProcessor-97445612619104267-127.0.0.1:56263_solr-n_0000000000) [n:127.0.0.1:56263_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2004129 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=acollectionafterbaddelete_shard1_replica1&action=CREATE&numShards=1&collection=acollectionafterbaddelete&shard=shard1&wt=javabin&version=2
[junit4] 2> 2004130 INFO (qtp1801337119-12724) [n:127.0.0.1:56265_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=acollectionafterbaddelete_shard1_replica2&action=CREATE&numShards=1&collection=acollectionafterbaddelete&shard=shard1&wt=javabin&version=2
[junit4] 2> 2004242 INFO (zkCallback-1961-thread-1-processing-n:127.0.0.1:56265_solr) [n:127.0.0.1:56265_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/acollectionafterbaddelete/state.json] for collection [acollectionafterbaddelete] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2004242 INFO (zkCallback-1959-thread-1-processing-n:127.0.0.1:56266_solr) [n:127.0.0.1:56266_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/acollectionafterbaddelete/state.json] for collection [acollectionafterbaddelete] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2005152 INFO (qtp1801337119-12724) [n:127.0.0.1:56265_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2005153 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2005160 INFO (qtp1801337119-12724) [n:127.0.0.1:56265_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica2] o.a.s.s.IndexSchema [acollectionafterbaddelete_shard1_replica2] Schema name=minimal
[junit4] 2> 2005160 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica1] o.a.s.s.IndexSchema [acollectionafterbaddelete_shard1_replica1] Schema name=minimal
[junit4] 2> 2005191 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2005192 INFO (qtp1801337119-12724) [n:127.0.0.1:56265_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2005194 INFO (qtp1801337119-12724) [n:127.0.0.1:56265_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica2] o.a.s.c.CoreContainer Creating SolrCore 'acollectionafterbaddelete_shard1_replica2' using configuration from collection acollectionafterbaddelete
[junit4] 2> 2005194 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'acollectionafterbaddelete_shard1_replica1' using configuration from collection acollectionafterbaddelete
[junit4] 2> 2005195 INFO (qtp1801337119-12724) [n:127.0.0.1:56265_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica2] o.a.s.c.SolrCore [[acollectionafterbaddelete_shard1_replica2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_31C36F6FB1443BEF-001/tempDir-001/node4/acollectionafterbaddelete_shard1_replica2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_31C36F6FB1443BEF-001/tempDir-001/node4/./acollectionafterbaddelete_shard1_replica2/data/]
[junit4] 2> 2005195 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica1] o.a.s.c.SolrCore [[acollectionafterbaddelete_shard1_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_31C36F6FB1443BEF-001/tempDir-001/node2/acollectionafterbaddelete_shard1_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_31C36F6FB1443BEF-001/tempDir-001/node2/./acollectionafterbaddelete_shard1_replica1/data/]
[junit4] 2> 2005296 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2005296 INFO (qtp1801337119-12724) [n:127.0.0.1:56265_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2005296 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2005296 INFO (qtp1801337119-12724) [n:127.0.0.1:56265_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2005299 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2005299 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2005299 INFO (qtp1801337119-12724) [n:127.0.0.1:56265_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2005299 INFO (qtp1801337119-12724) [n:127.0.0.1:56265_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2005300 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@21524a2[acollectionafterbaddelete_shard1_replica1] main]
[junit4] 2> 2005302 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2005303 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2005303 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2005303 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559129809400889344
[junit4] 2> 2005303 INFO (searcherExecutor-6035-thread-1-processing-n:127.0.0.1:56266_solr x:acollectionafterbaddelete_shard1_replica1 s:shard1 c:acollectionafterbaddelete) [n:127.0.0.1:56266_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica1] o.a.s.c.SolrCore [acollectionafterbaddelete_shard1_replica1] Registered new searcher Searcher@21524a2[acollectionafterbaddelete_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2005305 INFO (qtp1801337119-12724) [n:127.0.0.1:56265_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@60c08b7e[acollectionafterbaddelete_shard1_replica2] main]
[junit4] 2> 2005307 INFO (qtp1801337119-12724) [n:127.0.0.1:56265_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2005308 INFO (qtp1801337119-12724) [n:127.0.0.1:56265_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2005308 INFO (qtp1801337119-12724) [n:127.0.0.1:56265_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2005308 INFO (searcherExecutor-6034-thread-1-processing-n:127.0.0.1:56265_solr x:acollectionafterbaddelete_shard1_replica2 s:shard1 c:acollectionafterbaddelete) [n:127.0.0.1:56265_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica2] o.a.s.c.SolrCore [acollectionafterbaddelete_shard1_replica2] Registered new searcher Searcher@60c08b7e[acollectionafterbaddelete_shard1_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2005308 INFO (qtp1801337119-12724) [n:127.0.0.1:56265_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559129809406132224
[junit4] 2> 2005314 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2005314 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2005314 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:56266/solr/acollectionafterbaddelete_shard1_replica1/
[junit4] 2> 2005315 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica1] o.a.s.u.PeerSync PeerSync: core=acollectionafterbaddelete_shard1_replica1 url=http://127.0.0.1:56266/solr START replicas=[http://127.0.0.1:56265/solr/acollectionafterbaddelete_shard1_replica2/] nUpdates=100
[junit4] 2> 2005315 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica1] o.a.s.u.PeerSync PeerSync: core=acollectionafterbaddelete_shard1_replica1 url=http://127.0.0.1:56266/solr DONE. We have no versions. sync failed.
[junit4] 2> 2005317 INFO (qtp1801337119-12726) [n:127.0.0.1:56265_solr c:acollectionafterbaddelete s:shard1 r:core_node2 x:acollectionafterbaddelete_shard1_replica2] o.a.s.c.S.Request [acollectionafterbaddelete_shard1_replica2] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2005317 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 2005318 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 2005326 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:56266/solr/acollectionafterbaddelete_shard1_replica1/ shard1
[junit4] 2> 2005435 INFO (zkCallback-1961-thread-1-processing-n:127.0.0.1:56265_solr) [n:127.0.0.1:56265_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/acollectionafterbaddelete/state.json] for collection [acollectionafterbaddelete] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2005435 INFO (zkCallback-1959-thread-1-processing-n:127.0.0.1:56266_solr) [n:127.0.0.1:56266_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/acollectionafterbaddelete/state.json] for collection [acollectionafterbaddelete] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2005487 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2005490 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=acollectionafterbaddelete_shard1_replica1&action=CREATE&numShards=1&collection=acollectionafterbaddelete&shard=shard1&wt=javabin&version=2} status=0 QTime=1363
[junit4] 2> 2005603 INFO (zkCallback-1961-thread-1-processing-n:127.0.0.1:56265_solr) [n:127.0.0.1:56265_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/acollectionafterbaddelete/state.json] for collection [acollectionafterbaddelete] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2005603 INFO (zkCallback-1959-thread-1-processing-n:127.0.0.1:56266_solr) [n:127.0.0.1:56266_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/acollectionafterbaddelete/state.json] for collection [acollectionafterbaddelete] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2006321 INFO (qtp1801337119-12724) [n:127.0.0.1:56265_solr c:acollectionafterbaddelete s:shard1 x:acollectionafterbaddelete_shard1_replica2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=acollectionafterbaddelete_shard1_replica2&action=CREATE&numShards=1&collection=acollectionafterbaddelete&shard=shard1&wt=javabin&version=2} status=0 QTime=2191
[junit4] 2> 2006325 INFO (qtp1067139960-12742) [n:127.0.0.1:56266_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 2006428 INFO (zkCallback-1961-thread-1-processing-n:127.0.0.1:56265_solr) [n:127.0.0.1:56265_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/acollectionafterbaddelete/state.json] for collection [acollectionafterbaddelete] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2006428 INFO (zkCallback-1959-thread-1-processing-n:127.0.0.1:56266_solr) [n:127.0.0.1:56266_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/acollectionafterbaddelete/state.json] for collection [acollectionafterbaddelete] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2007329 INFO (qtp1067139960-12742) [n:127.0.0.1:56266_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=conf&name=acollectionafterbaddelete&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=3427
[junit4] 2> 2007332 INFO (TEST-CollectionsAPIDistributedZkTest.testDeleteNonExistentCollection-seed#[31C36F6FB1443BEF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testDeleteNonExistentCollection
[junit4] 2> 2007378 INFO (TEST-CollectionsAPIDistributedZkTest.deleteCollectionOnlyInZk-seed#[31C36F6FB1443BEF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting deleteCollectionOnlyInZk
[junit4] 2> 2007381 INFO (TEST-CollectionsAPIDistributedZkTest.deleteCollectionOnlyInZk-seed#[31C36F6FB1443BEF]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2007383 INFO (qtp1067139960-12738) [n:127.0.0.1:56266_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=acollectionafterbaddelete&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2007391 INFO (OverseerCollectionConfigSetProcessor-97445612619104267-127.0.0.1:56263_solr-n_0000000000) [n:127.0.0.1:56263_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2007391 INFO (OverseerThreadFactory-6020-thread-5-processing-n:127.0.0.1:56263_solr) [n:127.0.0.1:56263_solr ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
[junit4] 2> 2007393 INFO (qtp1067139960-12739) [n:127.0.0.1:56266_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.acollectionafterbaddelete.shard1.replica1
[junit4] 2> 2007393 INFO (qtp1067139960-12739) [n:127.0.0.1:56266_solr ] o.a.s.c.SolrCore [acollectionafterbaddelete_shard1_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@3cb8fdc3
[junit4] 2> 2007393 INFO (qtp1801337119-12732) [n:127.0.0.1:56265_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.acollectionafterbaddelete.shard1.replica2
[junit4] 2> 2007393 INFO (qtp1801337119-12732) [n:127.0.0.1:56265_solr ] o.a.s.c.SolrCore [acollectionafterbaddelete_shard1_replica2] CLOSING SolrCore org.apache.solr.core.SolrCore@f1a53e7
[junit4] 2> 2007394 INFO (qtp1067139960-12739) [n:127.0.0.1:56266_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.acollectionafterbaddelete.shard1.replica1
[junit4] 2> 2007394 INFO (qtp1801337119-12732) [n:127.0.0.1:56265_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.acollectionafterbaddelete.shard1.replica2
[junit4] 2> 2007399 WARN (zkCallback-1961-thread-1-processing-n:127.0.0.1:56265_solr) [n:127.0.0.1:56265_solr ] o.a.s.c.LeaderElector
[junit4] 2> org.apache.solr.common.SolrException: SolrCore not found:acollectionafterbaddelete_shard1_replica2 in []
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:292)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:56)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:348)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3.lambda$process$0(SolrZkClient.java:268)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 2007400 INFO (qtp1067139960-12739) [n:127.0.0.1:56266_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=acollectionafterbaddelete_shard1_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=7
[junit4] 2> 2007406 INFO (qtp1801337119-12732) [n:127.0.0.1:56265_solr ] o.a.s.c.ShardLeaderElectionContextBase No version found for ephemeral leader parent node, won't remove previous leader registration.
[junit4] 2> 2007406 INFO (qtp1801337119-12732) [n:127.0.0.1:56265_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=acollectionafterbaddelete_shard1_replica2&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=13
[junit4] 2> 2008132 INFO (qtp1067139960-12738) [n:127.0.0.1:56266_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=acollectionafterbaddelete&action=DELETE&wt=javabin&version=2} status=0 QTime=748
[junit4] 2> 2008137 INFO (qtp1067139960-12740) [n:127.0.0.1:56266_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=onlyinzk&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2008141 INFO (OverseerCollectionConfigSetProcessor-97445612619104267-127.0.0.1:56263_solr-n_0000000000) [n:127.0.0.1:56263_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000008 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2008147 INFO (qtp1067139960-12740) [n:127.0.0.1:56266_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=onlyinzk&action=DELETE&wt=javabin&version=2} status=0 QTime=10
[junit4] 2> 2008147 INFO (qtp1067139960-12741) [n:127.0.0.1:56266_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2008148 INFO (qtp1067139960-12741) [n:127.0.0.1:56266_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2008148 INFO (qtp1067139960-12720) [n:127.0.0.1:56266_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=onlyinzk&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2008152 INFO (OverseerThreadFactory-6020-thread-5-processing-n:127.0.0.1:56263_solr) [n:127.0.0.1:56263_solr ] o.a.s.c.CreateCollectionCmd Create collection onlyinzk
[junit4] 2> 2008153 INFO (OverseerCollectionConfigSetProcessor-97445612619104267-127.0.0.1:56263_solr-n_0000000000) [n:127.0.0.1:56263_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000010 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2008379 INFO (qtp1715910283-12717) [n:127.0.0.1:56263_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=onlyinzk_shard1_replica1&action=CREATE&numShards=2&collection=onlyinzk&shard=shard1&wt=javabin&version=2
[junit4] 2> 2008379 INFO (qtp1431369581-12728) [n:127.0.0.1:56264_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=onlyinzk_shard2_replica1&action=CREATE&numShards=2&collection=onlyinzk&shard=shard2&wt=javabin&version=2
[junit4] 2> 2008501 INFO (zkCallback-1962-thread-1-processing-n:127.0.0.1:56263_solr) [n:127.0.0.1:56263_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2008501 INFO (zkCallback-1960-thread-1-processing-n:127.0.0.1:56264_solr) [n:127.0.0.1:56264_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2009407 INFO (qtp1431369581-12728) [n:127.0.0.1:56264_solr c:onlyinzk s:shard2 x:onlyinzk_shard2_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2009408 INFO (qtp1715910283-12717) [n:127.0.0.1:56263_solr c:onlyinzk s:shard1 x:onlyinzk_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2009415 INFO (qtp1431369581-12728) [n:127.0.0.1:56264_solr c:onlyinzk s:shard2 x:onlyinzk_shard2_replica1] o.a.s.s.IndexSchema [onlyinzk_shard2_replica1] Schema name=minimal
[junit4] 2> 2009416 INFO (qtp1715910283-12717) [n:127.0.0.1:56263_solr c:onlyinzk s:shard1 x:onlyinzk_shard1_replica1] o.a.s.s.IndexSchema [onlyinzk_shard1_replica1] Schema name=minimal
[junit4] 2> 2009419 INFO (qtp1715910283-12717) [n:127.0.0.1:56263_solr c:onlyinzk s:shard1 x:onlyinzk_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2009420 INFO (qtp1431369581-12728) [n:127.0.0.1:56264_solr c:onlyinzk s:shard2 x:onlyinzk_shard2_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2009421 INFO (qtp1431369581-12728) [n:127.0.0.1:56264_solr c:onlyinzk s:shard2 x:onlyinzk_shard2_replica1] o.a.s.c.CoreContainer Creating SolrCore 'onlyinzk_shard2_replica1' using configuration from collection onlyinzk
[junit4] 2> 2009422 INFO (qtp1715910283-12717) [n:127.0.0.1:56263_solr c:onlyinzk s:shard1 x:onlyinzk_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'onlyinzk_shard1_replica1' using configuration from collection onlyinzk
[junit4] 2> 2009422 INFO (qtp1431369581-12728) [n:127.0.0.1:56264_solr c:onlyinzk s:shard2 x:onlyinzk_shard2_replica1] o.a.s.c.SolrCore [[onlyinzk_shard2_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_31C36F6FB1443BEF-001/tempDir-001/node3/onlyinzk_shard2_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_31C36F6FB1443BEF-001/tempDir-001/node3/./onlyinzk_shard2_replica1/data/]
[junit4] 2> 2009422 INFO (qtp1715910283-12717) [n:127.0.0.1:56263_solr c:onlyinzk s:shard1 x:onlyinzk_shard1_replica1] o.a.s.c.SolrCore [[onlyinzk_shard1_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_31C36F6FB1443BEF-001/tempDir-001/node1/onlyinzk_shard1_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_31C36F6FB1443BEF-001/tempDir-001/node1/./onlyinzk_shard1_replica1/data/]
[junit4] 2> 2009647 INFO (qtp1715910283-12717) [n:127.0.0.1:56263_solr c:onlyinzk s:shard1 x:onlyinzk_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2009647 INFO (qtp1715910283-12717) [n:127.0.0.1:56263_solr c:onlyinzk s:shard1 x:onlyinzk_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2009648 INFO (qtp1431369581-12728) [n:127.0.0.1:56264_solr c:onlyinzk s:shard2 x:onlyinzk_shard2_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2009648 INFO (qtp1431369581-12728) [n:127.0.0.1:56264_solr c:onlyinzk s:shard2 x:onlyinzk_shard2_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2009649 INFO (qtp1715910283-12717) [n:127.0.0.1:56263_solr c:onlyinzk s:shard1 x:onlyinzk_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2009649 INFO (qtp1715910283-12717) [n:127.0.0.1:56263_solr c:onlyinzk s:shard1 x:onlyinzk_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2009720 INFO (qtp1431369581-12728) [n:127.0.0.1:56264_solr c:onlyinzk s:shard2 x:onlyinzk_shard2_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2009720 INFO (qtp1431369581-12728) [n:127.0.0.1:56264_solr c:onlyinzk s:shard2 x:onlyinzk_shard2_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2009773 INFO (qtp1715910283-12717) [n:127.0.0.1:56263_solr c:onlyinzk s:shard1 x:onlyinzk_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1bbb4a4d[onlyinzk_shard1_replica1] main]
[junit4] 2> 2009775 INFO (qtp1715910283-12717) [n:127.0.0.1:56263_solr c:onlyinzk s:shard1 x:onlyinzk_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2009775 INFO (qtp1715910283-12717) [n:127.0.0.1:56263_solr c:onlyinzk s:shard1 x:onlyinzk_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2009775 INFO (qtp1715910283-12717) [n:127.0.0.1:56263_solr c:onlyinzk s:shard1 x:onlyinzk_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2009776 INFO (qtp1715910283-12717) [n:127.0.0.1:56263_solr c:onlyinzk s:shard1 x:onlyinzk_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559129814091169792
[junit4] 2> 2009776 INFO (searcherExecutor-6045-thread-1-processing-n:127.0.0.1:56263_solr x:onlyinzk_shard1_replica1 s:shard1 c:onlyinzk) [n:127.0.0.1:56263_solr c:onlyinzk s:shard1 x:onlyinzk_shard1_replica1] o.a.s.c.SolrCore [onlyinzk_shard1_replica1] Registered new searcher Searcher@1bbb4a4d[onlyinzk_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2009783 INFO (qtp1431369581-12728) [n:127.0.0.1:56264_solr c:onlyinzk s:shard2 x:onlyinzk_shard2_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2a0f7745[onlyinzk_shard2_replica1] main]
[junit4] 2> 2009785 INFO (qtp1431369581-12728) [n:127.0.0.1:56264_solr c:onlyinzk s:shard2 x:onlyinzk_shard2_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2009786 INFO (qtp1431369581-12728) [n:127.0.0.1:56264_solr c:onlyinzk s:shard2 x:onlyinzk_shard2_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2009786 INFO (qtp1431369581-12728) [n:127.0.0.1:56264_solr c:onlyinzk s:shard2 x:onlyinzk_shard2_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2009786 INFO (searcherExecutor-6044-thread-1-processing-n:127.0.0.1:56264_solr x:onlyinzk_shard2_replica1 s:shard2 c:onlyinzk) [n:127.0.0.1:56264_solr c:onlyinzk s:shard2 x:onlyinzk_shard2_replica1] o.a.s.c.SolrCore [onlyinzk_shard2_replica1] Registered new searcher Searcher@2a0f7745[onlyinzk_shard2_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2009786 INFO (qtp1431369581-12728) [n:127.0.0.1:56264_solr c:onlyinzk s:shard2 x:onlyinzk_shard2_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559129814101655552
[junit4] 2> 2009787 INFO (qtp1715910283-12717) [n:127.0.0.1:56263_solr c:onlyinzk s:shard1 x:onlyinzk_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2009787 INFO (qtp1715910283-12717) [n:127.0.0.1:56263_solr c:onlyinzk s:shard1 x:onlyinzk_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2009787 INFO (qtp1715910283-12717) [n:127.0.0.1:56263_solr c:onlyinzk s:shard1 x:onlyinzk_shard1_replica1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:56263/solr/onlyinzk_shard1_replica1/
[junit4] 2> 2009787 INFO (qtp1715910283-12717) [n:127.0.0.1:56263_solr c:onlyinzk s:shard1 x:onlyinzk_shard1_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2009787 INFO (qtp1715910283-12717) [n:127.0.0.1:56263_solr c:onlyinzk s:shard1 x:onlyinzk_shard1_replica1] o.a.s.c.SyncStrategy http://127.0.0.1:56263/solr/onlyinzk_shard1_replica1/ has no replicas
[junit4] 2> 2009796 INFO (qtp1715910283-12717) [n:127.0.0.1:56263_solr c:onlyinzk s:shard1 x:onlyinzk_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:56263/solr/onlyinzk_shard1_replica1/ shard1
[junit4] 2> 2009798 INFO (qtp1431369581-12728) [n:127.0.0.1:56264_solr c:onlyinzk s:shard2 x:onlyinzk_shard2_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2009799 INFO (qtp1431369581-12728) [n:127.0.0.1:56264_solr c:onlyinzk s:shard2 x:onlyinzk_shard2_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2009799 INFO (qtp1431369581-12728) [n:127.0.0.1:56264_solr c:onlyinzk s:shard2 x:onlyinzk_shard2_replica1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:56264/solr/onlyinzk_shard2_replica1/
[junit4] 2> 2009799 INFO (qtp1431369581-12728) [n:127.0.0.1:56264_solr c:onlyinzk s:shard2 x:onlyinzk_shard2_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2009799 INFO (qtp1431369581-12728) [n:127.0.0.1:56264_solr c:onlyinzk s:shard2 x:onlyinzk_shard2_replica1] o.a.s.c.SyncStrategy http://127.0.0.1:56264/solr/onlyinzk_shard2_replica1/ has no replicas
[junit4] 2> 2009805 INFO (qtp1431369581-12728) [n:127.0.0.1:56264_solr c:onlyinzk s:shard2 x:onlyinzk_shard2_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:56264/solr/onlyinzk_shard2_replica1/ shard2
[junit4] 2> 2009911 INFO (zkCallback-1960-thread-1-processing-n:127.0.0.1:56264_solr) [n:127.0.0.1:56264_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2009911 INFO (zkCallback-1962-thread-2-processing-n:127.0.0.1:56263_solr) [n:127.0.0.1:56263_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2009960 INFO (qtp1715910283-12717) [n:127.0.0.1:56263_solr c:onlyinzk s:shard1 x:onlyinzk_shard1_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2009962 INFO (qtp1431369581-12728) [n:127.0.0.1:56264_solr c:onlyinzk s:shard2 x:onlyinzk_shard2_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2009965 INFO (qtp1715910283-12717) [n:127.0.0.1:56263_solr c:onlyinzk s:shard1 x:onlyinzk_shard1_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=onlyinzk_shard1_replica1&action=CREATE&numShards=2&collection=onlyinzk&shard=shard1&wt=javabin&version=2} status=0 QTime=1586
[junit4] 2> 2009965 INFO (qtp1431369581-12728) [n:127.0.0.1:56264_solr c:onlyinzk s:shard2 x:onlyinzk_shard2_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=onlyinzk_shard2_replica1&action=CREATE&numShards=2&collection=onlyinzk&shard=shard2&wt=javabin&version=2} status=0 QTime=1586
[junit4] 2> 2009969 INFO (qtp1067139960-12720) [n:127.0.0.1:56266_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 2010071 INFO (zkCallback-1960-thread-1-processing-n:127.0.0.1:56264_solr) [n:127.0.0.1:56264_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2010071 INFO (zkCallback-1962-thread-1-processing-n:127.0.0.1:56263_solr) [n:127.0.0.1:56263_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2010166 INFO (OverseerCollectionConfigSetProcessor-97445612619104267-127.0.0.1:56263_solr-n_0000000000) [n:127.0.0.1:56263_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000012 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2010972 INFO (qtp1067139960-12720) [n:127.0.0.1:56266_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&name=onlyinzk&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=2823
[junit4] 2> 2010973 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2010974 INFO (qtp1067139960-12737) [n:127.0.0.1:56266_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2010974 INFO (TEST-CollectionsAPIDistributedZkTest.deleteCollectionOnlyInZk-seed#[31C36F6FB1443BEF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending deleteCollectionOnlyInZk
[junit4] 2> 2011013 INFO (TEST-CollectionsAPIDistributedZkTest.testNoConfigSetExist-seed#[31C36F6FB1443BEF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNoConfigSetExist
[junit4] 2> 2011015 INFO (TEST-CollectionsAPIDistributedZkTest.testNoConfigSetExist-seed#[31C36F6FB1443BEF]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2011018 INFO (qtp1067139960-12742) [n:127.0.0.1:56266_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=onlyinzk&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2011022 INFO (OverseerThreadFactory-6020-thread-5-processing-n:127.0.0.1:56263_solr) [n:127.0.0.1:56263_solr ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
[junit4] 2> 2011023 INFO (qtp1715910283-12719) [n:127.0.0.1:56263_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.onlyinzk.shard1.replica1
[junit4] 2> 2011023 INFO (qtp1715910283-12719) [n:127.0.0.1:56263_solr ] o.a.s.c.SolrCore [onlyinzk_shard1_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@491440ce
[junit4] 2> 2011023 INFO (qtp1431369581-12729) [n:127.0.0.1:56264_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.onlyinzk.shard2.replica1
[junit4] 2> 2011023 INFO (qtp1431369581-12729) [n:127.0.0.1:56264_solr ] o.a.s.c.SolrCore [onlyinzk_shard2_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@391ca023
[junit4] 2> 2011024 INFO (qtp1715910283-12719) [n:127.0.0.1:56263_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.onlyinzk.shard1.replica1
[junit4] 2> 2011024 INFO (qtp1431369581-12729) [n:127.0.0.1:56264_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.onlyinzk.shard2.replica1
[junit4] 2> 2011058 INFO (qtp1715910283-12719) [n:127.0.0.1:56263_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=onlyinzk_shard1_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=35
[junit4] 2> 2011058 INFO (qtp1431369581-12729) [n:127.0.0.1:56264_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=onlyinzk_shard2_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=35
[junit4] 2> 2011783 INFO (qtp1067139960-12742) [n:127.0.0.1:56266_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=onlyinzk&action=DELETE&wt=javabin&version=2} status=0 QTime=765
[junit4] 2> 2011785 INFO (qtp1067139960-12739) [n:127.0.0.1:56266_solr ] o.a.s.h.a.CoreAdminOperation core create command dataDir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_31C36F6FB1443BEF-001/tempDir-002&collection.configName=conf123&name=corewithnocollection3&action=CREATE&numShards=1&collection=&wt=javabin&version=2
[junit4] 2> 2011788 INFO (OverseerStateUpdate-97445612619104267-127.0.0.1:56263_solr-n_0000000000) [n:127.0.0.1:56263_solr ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 2012796 ERROR (qtp1067139960-12739) [n:127.0.0.1:56266_solr c:corewithnocollection3 x:corewithnocollection3] o.a.s.c.c.ZkStateReader Specified config=[conf123] does not exist in ZooKeeper at location=[/configs/conf123]
[junit4] 2> 2012798 ERROR (qtp1067139960-12739) [n:127.0.0.1:56266_solr c:corewithnocollection3 x:corewithnocollection3] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Error CREATEing SolrCore 'corewithnocollection3': Unable to create core [corewithnocollection3] Caused by: Specified config does not exist in ZooKeeper: conf123
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:907)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:88)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:378)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:388)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:171)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:747)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:728)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:509)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:298)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:462)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> Caused by: org.apache.solr.common.SolrException: Unable to create core [corewithnocollection3]
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:952)
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:873)
[junit4] 2> ... 34 more
[junit4] 2> Caused by: org.apache.solr.common.cloud.ZooKeeperException: Specified config does not exist in ZooKeeper: conf123
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.readConfigName(ZkStateReader.java:189)
[junit4] 2> at org.apache.solr.cloud.CloudConfigSetService.createCoreResourceLoader(CloudConfigSetService.java:54)
[junit4] 2> at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:74)
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:934)
[junit4] 2> ... 35 more
[junit4] 2>
[junit4] 2> 2012798 INFO (qtp1067139960-12739) [n:127.0.0.1:56266_solr c:corewithnocollection3 x:corewithnocollection3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={dataDir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_31C36F6FB1443BEF-001/tempDir-002&collection.configName=conf123&name=corewithnocollection3&action=CREATE&numShards=1&collection=&wt=javabin&version=2} status=400 QTime=1013
[junit4] 2> 2013025 INFO (OverseerCollectionConfigSetProcessor-97445612619104267-127.0.0.1:56263_solr-n_0000000000) [n:127.0.0.1:56263_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000014 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2013811 INFO (qtp1067139960-12738) [n:127.0.0.1:56266_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2013811 INFO (qtp1067139960-12738) [n:127.0.0.1:56266_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2013811 INFO (TEST-CollectionsAPIDistributedZkTest.testNoConfigSetExist-seed#[31C36F6FB1443BEF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNoConfigSetExist
[junit4] 2> 2013847 INFO (TEST-CollectionsAPIDistributedZkTest.addReplicaTest-seed#[31C36F6FB1443BEF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting addReplicaTest
[junit4] 2> 2013850 INFO (TEST-CollectionsAPIDistributedZkTest.addReplicaTest-seed#[31C36F6FB1443BEF]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2013852 INFO (qtp1067139960-12740) [n:127.0.0.1:56266_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=corewithnocollection3&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2013856 INFO (OverseerThreadFactory-6020-thread-5-processing-n:127.0.0.1:56263_solr) [n:127.0.0.1:56263_solr ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
[junit4] 2> 2014586 INFO (qtp1067139960-12740) [n:127.0.0.1:56266_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=corewithnocollection3&action=DELETE&wt=javabin&version=2} status=0 QTime=733
[junit4] 2> 2014587 INFO (qtp1067139960-12741) [n:127.0.0.1:56266_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&maxShardsPerNode=4&name=addReplicaColl&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2014591 INFO (OverseerThreadFactory-6020-thread-5-processing-n:127.0.0.1:56263_solr) [n:127.0.0.1:56263_solr ] o.a.s.c.CreateCollectionCmd Create collection addReplicaColl
[junit4] 2> 2014593 INFO (OverseerCollectionConfigSetProcessor-97445612619104267-127.0.0.1:56263_solr-n_0000000000) [n:127.0.0.1:56263_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000016 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2014825 INFO (qtp1431369581-12730) [n:127.0.0.1:56264_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=addReplicaColl_shard1_replica2&action=CREATE&numShards=2&collection=addReplicaColl&shard=shard1&wt=javabin&version=2
[junit4] 2> 2014826 INFO (qtp1715910283-12721) [n:127.0.0.1:56263_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=addReplicaColl_shard2_replica2&action=CREATE&numShards=2&collection=addReplicaColl&shard=shard2&wt=javabin&version=2
[junit4] 2> 2014827 INFO (qtp1067139960-12720) [n:127.0.0.1:56266_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=addReplicaColl_shard1_replica1&action=CREATE&numShards=2&collection=addReplicaColl&shard=shard1&wt=javabin&version=2
[junit4] 2> 2014831 INFO (qtp1801337119-12743) [n:127.0.0.1:56265_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=addReplicaColl_shard2_replica1&action=CREATE&numShards=2&collection=addReplicaColl&shard=shard2&wt=javabin&version=2
[junit4] 2> 2014954 INFO (zkCallback-1962-thread-3-processing-n:127.0.0.1:56263_solr) [n:127.0.0.1:56263_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2014954 INFO (zkCallback-1960-thread-2-processing-n:127.0.0.1:56264_solr) [n:127.0.0.1:56264_solr ] o.a.s.c.c.ZkStateReader A c
[...truncated too long message...]
(zkCallback-1962-thread-19-processing-n:127.0.0.1:56263_solr) [n:127.0.0.1:56263_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (1)
[junit4] 2> 2177608 INFO (jetty-launcher-1941-thread-8) [ ] o.a.s.c.Overseer Overseer (id=97445612619104267-127.0.0.1:56263_solr-n_0000000000) closing
[junit4] 2> 2177611 INFO (OverseerStateUpdate-97445612619104267-127.0.0.1:56263_solr-n_0000000000) [n:127.0.0.1:56263_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:56263_solr
[junit4] 2> 2179103 WARN (zkCallback-1959-thread-4-processing-n:127.0.0.1:56266_solr) [n:127.0.0.1:56266_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 2179103 INFO (jetty-launcher-1941-thread-7) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 2179104 INFO (jetty-launcher-1941-thread-7) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5723c47a{/solr,null,UNAVAILABLE}
[junit4] 2> 2179115 WARN (zkCallback-1962-thread-19-processing-n:127.0.0.1:56263_solr) [n:127.0.0.1:56263_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 2179115 INFO (jetty-launcher-1941-thread-8) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 2179115 INFO (jetty-launcher-1941-thread-8) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@22099fb3{/solr,null,UNAVAILABLE}
[junit4] 2> 2179122 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[31C36F6FB1443BEF]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:56261 56261
[junit4] 2> 2180287 INFO (Thread-2725) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:56261 56261
[junit4] 2> 2183671 WARN (Thread-2725) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 25 /solr/aliases.json
[junit4] 2> 25 /solr/clusterprops.json
[junit4] 2> 23 /solr/configs/conf
[junit4] 2> 4 /solr/security.json
[junit4] 2> 4 /solr/configs/conf2
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 275 /solr/clusterstate.json
[junit4] 2> 5 /solr/collections/awhollynewcollection_0/state.json
[junit4] 2> 4 /solr/collections/nodes_used_collection/state.json
[junit4] 2> 4 /solr/collections/addReplicaColl/state.json
[junit4] 2> 4 /solr/collections/reloaded_collection/state.json
[junit4] 2> 3 /solr/collections/acollectionafterbaddelete/state.json
[junit4] 2> 2 /solr/collections/awhollynewcollection_0/leader_elect/shard3/election/97445612619104266-core_node6-n_0000000000
[junit4] 2> 2 /solr/collections/onlyinzk/state.json
[junit4] 2> 2 /solr/collections/halfdeletedcollection/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 380 /solr/collections
[junit4] 2> 181 /solr/overseer/queue
[junit4] 2> 111 /solr/overseer/collection-queue-work
[junit4] 2> 25 /solr/live_nodes
[junit4] 2> 22 /solr/overseer/queue-work
[junit4] 2>
[junit4] 2> 2183672 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[31C36F6FB1443BEF]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_31C36F6FB1443BEF-001
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=true): {}, locale=sr-Latn-BA, timezone=America/Argentina/Rio_Gallegos
[junit4] 2> NOTE: Mac OS X 10.11.6 x86_64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=3,threads=1,free=243789400,total=518979584
[junit4] 2> NOTE: All tests run in this JVM: [TestDistributedStatsComponentCardinality, ManagedSchemaRoundRobinCloudTest, HdfsNNFailoverTest, TestWriterPerf, TestRandomFaceting, TestPivotHelperCode, AnalysisErrorHandlingTest, NoCacheHeaderTest, SuggesterWFSTTest, TestSQLHandler, CoreAdminHandlerTest, TestGeoJSONResponseWriter, RuleEngineTest, TestQueryWrapperFilter, TestCloudPseudoReturnFields, ZkSolrClientTest, TestManagedSchemaThreadSafety, TestFreeTextSuggestions, TestFastWriter, MinimalSchemaTest, UpdateRequestProcessorFactoryTest, ReturnFieldsTest, WordBreakSolrSpellCheckerTest, SignatureUpdateProcessorFactoryTest, TestFieldSortValues, TestIndexSearcher, UpdateParamsTest, TestTestInjection, BlockJoinFacetRandomTest, TestExceedMaxTermLength, RAMDirectoryFactoryTest, BadIndexSchemaTest, TestRandomFlRTGCloud, TestCloudSchemaless, TestFoldingMultitermQuery, TestFuzzyAnalyzedSuggestions, TestOrdValues, HighlighterConfigTest, DistributedTermsComponentTest, TestStressInPlaceUpdates, TestConfig, DistributedSuggestComponentTest, CircularListTest, DistanceFunctionTest, TestManagedSchema, SuggestComponentContextFilterQueryTest, TestSolrConfigHandler, TestConfigSetsAPI, TestConfigSetsAPIZkFailure, CachingDirectoryFactoryTest, TestRecovery, TestAnalyzedSuggestions, TestReversedWildcardFilterFactory, TestShortCircuitedRequests, TestRealTimeGet, DistributedQueryComponentCustomSortTest, CloudMLTQParserTest, TestIndexingPerformance, ActionThrottleTest, ShardRoutingCustomTest, DirectUpdateHandlerOptimizeTest, GraphQueryTest, VMParamsZkACLAndCredentialsProvidersTest, ClassificationUpdateProcessorTest, TestManagedResourceStorage, TestDistribIDF, SmileWriterTest, TestReqParamsAPI, UUIDUpdateProcessorFallbackTest, CollectionsAPIAsyncDistributedZkTest, BaseCdcrDistributedZkTest, TestValueSourceCache, HdfsWriteToMultipleCollectionsTest, TestSolrQueryResponse, LoggingHandlerTest, CustomCollectionTest, PeerSyncReplicationTest, TestQueryUtils, ExternalFileFieldSortTest, TestJmxMonitoredMap, ChangedSchemaMergeTest, TestPathTrie, TestFieldCollectionResource, TestSSLRandomization, TestSolr4Spatial, TestPKIAuthenticationPlugin, SortByFunctionTest, HdfsSyncSliceTest, TestReloadAndDeleteDocs, RankQueryTest, TestNumericTerms32, SolrIndexSplitterTest, HdfsBasicDistributedZkTest, TestSolrCoreParser, SliceStateTest, DocExpirationUpdateProcessorFactoryTest, TestGraphTermsQParserPlugin, TestCloudManagedSchema, TestSolrCoreProperties, LeaderFailoverAfterPartitionTest, TestExactStatsCache, TestSuggestSpellingConverter, CacheHeaderTest, DistributedFacetPivotSmallTest, WrapperMergePolicyFactoryTest, TestInitParams, TestComplexPhraseQParserPlugin, CoreAdminCreateDiscoverTest, TestZkChroot, TestSearchPerf, HdfsTlogReplayBufferedWhileIndexingTest, BasicZkTest, TestImpersonationWithHadoopAuth, JsonLoaderTest, TestCodecSupport, FullSolrCloudDistribCmdsTest, TestSha256AuthenticationProvider, TestXmlQParserPlugin, SegmentsInfoRequestHandlerTest, TestJettySolrRunner, CdcrReplicationDistributedZkTest, ChaosMonkeySafeLeaderTest, AutoCommitTest, CurrencyFieldOpenExchangeTest, DirectoryFactoryTest, SpellingQueryConverterTest, SpellCheckCollatorTest, TestConfigSetImmutable, MultiThreadedOCPTest, TestQueryTypes, TestSimpleTrackingShardHandler, TestConfigSets, TestSystemCollAutoCreate, TestCopyFieldCollectionResource, TestInPlaceUpdatesDistrib, TestManagedStopFilterFactory, TestSubQueryTransformerDistrib, TestStandardQParsers, TestOmitPositions, TestJsonFacets, BlobRepositoryCloudTest, TestAuthenticationFramework, MigrateRouteKeyTest, RemoteQueryErrorTest, DeleteLastCustomShardedReplicaTest, ConcurrentDeleteAndCreateCollectionTest, TestCollapseQParserPlugin, TestSchemaNameResource, TermVectorComponentDistributedTest, TlogReplayBufferedWhileIndexingTest, TestLMJelinekMercerSimilarityFactory, TestRTimerTree, URLClassifyProcessorTest, SolrCoreCheckLockOnStartupTest, TestCustomDocTransformer, TestLuceneMatchVersion, TestCloudInspectUtil, TestSolrCloudWithDelegationTokens, ResponseLogComponentTest, TestFieldCacheSortRandom, TestSolrCloudWithSecureImpersonation, TestJavabinTupleStreamParser, TermVectorComponentTest, HdfsRecoveryZkTest, HdfsChaosMonkeySafeLeaderTest, DistributedFacetPivotLongTailTest, TestBulkSchemaAPI, TestUnifiedSolrHighlighter, RollingRestartTest, BasicDistributedZkTest, BasicDistributedZk2Test, CollectionsAPIDistributedZkTest]
[junit4] Completed [342/694 (1!)] on J1 in 185.55s, 20 tests, 1 failure <<< FAILURES!
[...truncated 63342 lines...]
-documentation-lint:
[jtidy] Checking for broken html (such as invalid tags)...
[delete] Deleting directory /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/jtidy_tmp
[echo] Checking for broken links...
[exec]
[exec] Crawl/parse...
[exec]
[exec] Verify...
[echo] Checking for malformed docs...
[exec]
[exec] /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/docs/solr-solrj/overview-summary.html
[exec] missing: org.apache.solr.client.solrj.io.eval
[exec]
[exec] Missing javadocs were found!
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/build.xml:775: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/build.xml:101: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build.xml:654: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build.xml:670: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/common-build.xml:2493: exec returned: 1
Total time: 116 minutes 18 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3826
- Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3826/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
2 tests failed.
FAILED: org.apache.solr.cloud.CustomCollectionTest.testRouteFieldForImplicitRouter
Error Message:
Collection not found: withShardField
Stack Trace:
org.apache.solr.common.SolrException: Collection not found: withShardField
at __randomizedtesting.SeedInfo.seed([A23057AEB5080A8B:F760BF3C19F1C57B]:0)
at org.apache.solr.client.solrj.impl.CloudSolrClient.getCollectionNames(CloudSolrClient.java:1376)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1072)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1042)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:232)
at org.apache.solr.cloud.CustomCollectionTest.testRouteFieldForImplicitRouter(CustomCollectionTest.java:141)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.PeerSyncReplicationTest.test
Error Message:
timeout waiting to see all nodes active
Stack Trace:
java.lang.AssertionError: timeout waiting to see all nodes active
at __randomizedtesting.SeedInfo.seed([A23057AEB5080A8B:2A6468741BF46773]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.PeerSyncReplicationTest.waitTillNodesActive(PeerSyncReplicationTest.java:326)
at org.apache.solr.cloud.PeerSyncReplicationTest.bringUpDeadNodeAndEnsureNoReplication(PeerSyncReplicationTest.java:277)
at org.apache.solr.cloud.PeerSyncReplicationTest.forceNodeFailureAndDoPeerSync(PeerSyncReplicationTest.java:259)
at org.apache.solr.cloud.PeerSyncReplicationTest.test(PeerSyncReplicationTest.java:138)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11649 lines...]
[junit4] Suite: org.apache.solr.cloud.CustomCollectionTest
[junit4] 2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_A23057AEB5080A8B-001/init-core-data-001
[junit4] 2> 1750172 INFO (SUITE-CustomCollectionTest-seed#[A23057AEB5080A8B]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 1750176 INFO (SUITE-CustomCollectionTest-seed#[A23057AEB5080A8B]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN) w/ MAC_OS_X supressed clientAuth
[junit4] 2> 1750177 INFO (SUITE-CustomCollectionTest-seed#[A23057AEB5080A8B]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_A23057AEB5080A8B-001/tempDir-001
[junit4] 2> 1750177 INFO (SUITE-CustomCollectionTest-seed#[A23057AEB5080A8B]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1750177 INFO (Thread-2364) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1750177 INFO (Thread-2364) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1750287 INFO (SUITE-CustomCollectionTest-seed#[A23057AEB5080A8B]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:52870
[junit4] 2> 1750318 INFO (jetty-launcher-2834-thread-2) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1750318 INFO (jetty-launcher-2834-thread-1) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1750318 INFO (jetty-launcher-2834-thread-3) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1750319 INFO (jetty-launcher-2834-thread-4) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1750322 INFO (jetty-launcher-2834-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@63b610e2{/solr,null,AVAILABLE}
[junit4] 2> 1750323 INFO (jetty-launcher-2834-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@833de9d{/solr,null,AVAILABLE}
[junit4] 2> 1750324 INFO (jetty-launcher-2834-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@147171d2{HTTP/1.1,[http/1.1]}{127.0.0.1:52872}
[junit4] 2> 1750324 INFO (jetty-launcher-2834-thread-2) [ ] o.e.j.s.Server Started @1756626ms
[junit4] 2> 1750324 INFO (jetty-launcher-2834-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=52872}
[junit4] 2> 1750324 ERROR (jetty-launcher-2834-thread-2) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1750324 INFO (jetty-launcher-2834-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1256471b{/solr,null,AVAILABLE}
[junit4] 2> 1750325 INFO (jetty-launcher-2834-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 1750325 INFO (jetty-launcher-2834-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3cd0af62{HTTP/1.1,[http/1.1]}{127.0.0.1:52874}
[junit4] 2> 1750325 INFO (jetty-launcher-2834-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1750325 INFO (jetty-launcher-2834-thread-4) [ ] o.e.j.s.Server Started @1756628ms
[junit4] 2> 1750325 INFO (jetty-launcher-2834-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1750325 INFO (jetty-launcher-2834-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-12T00:08:51.683Z
[junit4] 2> 1750325 INFO (jetty-launcher-2834-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=52874}
[junit4] 2> 1750326 ERROR (jetty-launcher-2834-thread-4) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1750326 INFO (jetty-launcher-2834-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 1750326 INFO (jetty-launcher-2834-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1750326 INFO (jetty-launcher-2834-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1750326 INFO (jetty-launcher-2834-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-12T00:08:51.684Z
[junit4] 2> 1750329 INFO (jetty-launcher-2834-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@55355f64{/solr,null,AVAILABLE}
[junit4] 2> 1750330 INFO (jetty-launcher-2834-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2a7bf5a8{HTTP/1.1,[http/1.1]}{127.0.0.1:52875}
[junit4] 2> 1750330 INFO (jetty-launcher-2834-thread-1) [ ] o.e.j.s.Server Started @1756633ms
[junit4] 2> 1750331 INFO (jetty-launcher-2834-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@c4643d8{HTTP/1.1,[http/1.1]}{127.0.0.1:52873}
[junit4] 2> 1750331 INFO (jetty-launcher-2834-thread-3) [ ] o.e.j.s.Server Started @1756634ms
[junit4] 2> 1750331 INFO (jetty-launcher-2834-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=52875}
[junit4] 2> 1750332 ERROR (jetty-launcher-2834-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1750332 INFO (jetty-launcher-2834-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=52873}
[junit4] 2> 1750333 ERROR (jetty-launcher-2834-thread-3) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1750333 INFO (jetty-launcher-2834-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 1750333 INFO (jetty-launcher-2834-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1750333 INFO (jetty-launcher-2834-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1750333 INFO (jetty-launcher-2834-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-12T00:08:51.691Z
[junit4] 2> 1750334 INFO (jetty-launcher-2834-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 1750334 INFO (jetty-launcher-2834-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1750334 INFO (jetty-launcher-2834-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1750334 INFO (jetty-launcher-2834-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-12T00:08:51.692Z
[junit4] 2> 1750338 INFO (jetty-launcher-2834-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1750338 INFO (jetty-launcher-2834-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1750339 INFO (jetty-launcher-2834-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1750340 INFO (jetty-launcher-2834-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1750353 INFO (jetty-launcher-2834-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52870/solr
[junit4] 2> 1750355 INFO (jetty-launcher-2834-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52870/solr
[junit4] 2> 1750357 INFO (jetty-launcher-2834-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52870/solr
[junit4] 2> 1750360 INFO (jetty-launcher-2834-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52870/solr
[junit4] 2> 1750430 INFO (jetty-launcher-2834-thread-1) [n:127.0.0.1:52875_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:52875_solr
[junit4] 2> 1750432 INFO (jetty-launcher-2834-thread-3) [n:127.0.0.1:52873_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52873_solr
[junit4] 2> 1750432 INFO (jetty-launcher-2834-thread-1) [n:127.0.0.1:52875_solr ] o.a.s.c.Overseer Overseer (id=97442734508474377-127.0.0.1:52875_solr-n_0000000000) starting
[junit4] 2> 1750433 INFO (jetty-launcher-2834-thread-4) [n:127.0.0.1:52874_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52874_solr
[junit4] 2> 1750436 INFO (zkCallback-2852-thread-1-processing-n:127.0.0.1:52875_solr) [n:127.0.0.1:52875_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1750437 INFO (zkCallback-2855-thread-1-processing-n:127.0.0.1:52874_solr) [n:127.0.0.1:52874_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1750438 INFO (zkCallback-2853-thread-1-processing-n:127.0.0.1:52873_solr) [n:127.0.0.1:52873_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1750438 INFO (zkCallback-2854-thread-1-processing-n:127.0.0.1:52872_solr) [n:127.0.0.1:52872_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1750441 INFO (zkCallback-2852-thread-1-processing-n:127.0.0.1:52875_solr) [n:127.0.0.1:52875_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1750441 INFO (zkCallback-2855-thread-1-processing-n:127.0.0.1:52874_solr) [n:127.0.0.1:52874_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1750442 INFO (zkCallback-2853-thread-1-processing-n:127.0.0.1:52873_solr) [n:127.0.0.1:52873_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1750443 INFO (zkCallback-2854-thread-1-processing-n:127.0.0.1:52872_solr) [n:127.0.0.1:52872_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1750463 INFO (jetty-launcher-2834-thread-1) [n:127.0.0.1:52875_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52875_solr
[junit4] 2> 1750467 INFO (zkCallback-2852-thread-1-processing-n:127.0.0.1:52875_solr) [n:127.0.0.1:52875_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1750467 INFO (zkCallback-2855-thread-1-processing-n:127.0.0.1:52874_solr) [n:127.0.0.1:52874_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1750468 INFO (zkCallback-2853-thread-1-processing-n:127.0.0.1:52873_solr) [n:127.0.0.1:52873_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1750468 INFO (zkCallback-2854-thread-1-processing-n:127.0.0.1:52872_solr) [n:127.0.0.1:52872_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1750472 INFO (jetty-launcher-2834-thread-2) [n:127.0.0.1:52872_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52872_solr
[junit4] 2> 1750478 INFO (zkCallback-2853-thread-1-processing-n:127.0.0.1:52873_solr) [n:127.0.0.1:52873_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1750478 INFO (zkCallback-2855-thread-1-processing-n:127.0.0.1:52874_solr) [n:127.0.0.1:52874_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1750478 INFO (zkCallback-2852-thread-1-processing-n:127.0.0.1:52875_solr) [n:127.0.0.1:52875_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1750479 INFO (zkCallback-2854-thread-1-processing-n:127.0.0.1:52872_solr) [n:127.0.0.1:52872_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1750556 INFO (jetty-launcher-2834-thread-3) [n:127.0.0.1:52873_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_A23057AEB5080A8B-001/tempDir-001/node3/.
[junit4] 2> 1750563 INFO (jetty-launcher-2834-thread-1) [n:127.0.0.1:52875_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_A23057AEB5080A8B-001/tempDir-001/node1/.
[junit4] 2> 1750570 INFO (jetty-launcher-2834-thread-4) [n:127.0.0.1:52874_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_A23057AEB5080A8B-001/tempDir-001/node4/.
[junit4] 2> 1750621 INFO (jetty-launcher-2834-thread-2) [n:127.0.0.1:52872_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_A23057AEB5080A8B-001/tempDir-001/node2/.
[junit4] 2> 1751909 INFO (SUITE-CustomCollectionTest-seed#[A23057AEB5080A8B]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1751911 INFO (SUITE-CustomCollectionTest-seed#[A23057AEB5080A8B]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:52870/solr ready
[junit4] 2> 1752003 INFO (TEST-CustomCollectionTest.testRouteFieldForHashRouter-seed#[A23057AEB5080A8B]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testRouteFieldForHashRouter
[junit4] 2> 1752005 INFO (TEST-CustomCollectionTest.testRouteFieldForHashRouter-seed#[A23057AEB5080A8B]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1752010 INFO (qtp1239476393-13375) [n:127.0.0.1:52874_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&maxShardsPerNode=3&router.field=shard_s&name=routeFieldColl&action=CREATE&numShards=4&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1752013 INFO (OverseerThreadFactory-5528-thread-1-processing-n:127.0.0.1:52875_solr) [n:127.0.0.1:52875_solr ] o.a.s.c.CreateCollectionCmd Create collection routeFieldColl
[junit4] 2> 1752138 INFO (qtp1239476393-13377) [n:127.0.0.1:52874_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=routeFieldColl_shard4_replica1&action=CREATE&numShards=4&collection=routeFieldColl&shard=shard4&wt=javabin&version=2
[junit4] 2> 1752138 INFO (qtp967218453-13366) [n:127.0.0.1:52873_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=routeFieldColl_shard3_replica1&action=CREATE&numShards=4&collection=routeFieldColl&shard=shard3&wt=javabin&version=2
[junit4] 2> 1752138 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=routeFieldColl_shard1_replica2&action=CREATE&numShards=4&collection=routeFieldColl&shard=shard1&wt=javabin&version=2
[junit4] 2> 1752139 INFO (qtp1239476393-13379) [n:127.0.0.1:52874_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=routeFieldColl_shard2_replica1&action=CREATE&numShards=4&collection=routeFieldColl&shard=shard2&wt=javabin&version=2
[junit4] 2> 1752139 INFO (qtp967218453-13368) [n:127.0.0.1:52873_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=routeFieldColl_shard1_replica1&action=CREATE&numShards=4&collection=routeFieldColl&shard=shard1&wt=javabin&version=2
[junit4] 2> 1752140 INFO (qtp582900176-13355) [n:127.0.0.1:52872_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=routeFieldColl_shard2_replica2&action=CREATE&numShards=4&collection=routeFieldColl&shard=shard2&wt=javabin&version=2
[junit4] 2> 1752141 INFO (qtp1958637594-13385) [n:127.0.0.1:52875_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=routeFieldColl_shard3_replica2&action=CREATE&numShards=4&collection=routeFieldColl&shard=shard3&wt=javabin&version=2
[junit4] 2> 1752144 INFO (qtp582900176-13357) [n:127.0.0.1:52872_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=routeFieldColl_shard4_replica2&action=CREATE&numShards=4&collection=routeFieldColl&shard=shard4&wt=javabin&version=2
[junit4] 2> 1752283 INFO (zkCallback-2852-thread-1-processing-n:127.0.0.1:52875_solr) [n:127.0.0.1:52875_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routeFieldColl/state.json] for collection [routeFieldColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1752284 INFO (zkCallback-2853-thread-1-processing-n:127.0.0.1:52873_solr) [n:127.0.0.1:52873_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routeFieldColl/state.json] for collection [routeFieldColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1752283 INFO (zkCallback-2855-thread-1-processing-n:127.0.0.1:52874_solr) [n:127.0.0.1:52874_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routeFieldColl/state.json] for collection [routeFieldColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1752284 INFO (zkCallback-2854-thread-1-processing-n:127.0.0.1:52872_solr) [n:127.0.0.1:52872_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routeFieldColl/state.json] for collection [routeFieldColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1753181 INFO (qtp1239476393-13379) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1753181 INFO (qtp1958637594-13385) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1753188 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1753201 INFO (qtp967218453-13366) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1753202 INFO (qtp582900176-13355) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1753209 INFO (qtp1239476393-13377) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1753210 INFO (qtp582900176-13357) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1753225 INFO (qtp967218453-13368) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1753234 INFO (qtp967218453-13366) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.s.IndexSchema [routeFieldColl_shard3_replica1] Schema name=cloud-dynamic
[junit4] 2> 1753234 INFO (qtp1239476393-13379) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.s.IndexSchema [routeFieldColl_shard2_replica1] Schema name=cloud-dynamic
[junit4] 2> 1753236 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.s.IndexSchema [routeFieldColl_shard1_replica2] Schema name=cloud-dynamic
[junit4] 2> 1753240 INFO (qtp1958637594-13385) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.s.IndexSchema [routeFieldColl_shard3_replica2] Schema name=cloud-dynamic
[junit4] 2> 1753261 INFO (qtp582900176-13357) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica2] o.a.s.s.IndexSchema [routeFieldColl_shard4_replica2] Schema name=cloud-dynamic
[junit4] 2> 1753280 INFO (qtp1239476393-13377) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.s.IndexSchema [routeFieldColl_shard4_replica1] Schema name=cloud-dynamic
[junit4] 2> 1753286 INFO (qtp582900176-13355) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.s.IndexSchema [routeFieldColl_shard2_replica2] Schema name=cloud-dynamic
[junit4] 2> 1753290 WARN (qtp967218453-13366) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.s.IndexSchema [routeFieldColl_shard3_replica1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1753291 INFO (qtp967218453-13366) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 1753304 WARN (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.s.IndexSchema [routeFieldColl_shard1_replica2] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1753304 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 1753311 WARN (qtp582900176-13357) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica2] o.a.s.s.IndexSchema [routeFieldColl_shard4_replica2] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1753312 INFO (qtp582900176-13357) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica2] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 1753313 WARN (qtp1239476393-13379) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.s.IndexSchema [routeFieldColl_shard2_replica1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1753314 INFO (qtp1239476393-13379) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 1753316 WARN (qtp1958637594-13385) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.s.IndexSchema [routeFieldColl_shard3_replica2] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1753317 INFO (qtp1958637594-13385) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 1753336 INFO (qtp582900176-13357) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica2] o.a.s.c.CoreContainer Creating SolrCore 'routeFieldColl_shard4_replica2' using configuration from collection routeFieldColl
[junit4] 2> 1753336 INFO (qtp582900176-13357) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica2] o.a.s.c.SolrCore [[routeFieldColl_shard4_replica2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_A23057AEB5080A8B-001/tempDir-001/node2/routeFieldColl_shard4_replica2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_A23057AEB5080A8B-001/tempDir-001/node2/./routeFieldColl_shard4_replica2/data/]
[junit4] 2> 1753338 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.c.CoreContainer Creating SolrCore 'routeFieldColl_shard1_replica2' using configuration from collection routeFieldColl
[junit4] 2> 1753338 WARN (qtp1239476393-13377) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.s.IndexSchema [routeFieldColl_shard4_replica1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1753338 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.c.SolrCore [[routeFieldColl_shard1_replica2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_A23057AEB5080A8B-001/tempDir-001/node1/routeFieldColl_shard1_replica2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_A23057AEB5080A8B-001/tempDir-001/node1/./routeFieldColl_shard1_replica2/data/]
[junit4] 2> 1753339 INFO (qtp1239476393-13377) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 1753339 INFO (qtp1958637594-13385) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.c.CoreContainer Creating SolrCore 'routeFieldColl_shard3_replica2' using configuration from collection routeFieldColl
[junit4] 2> 1753339 INFO (qtp1958637594-13385) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.c.SolrCore [[routeFieldColl_shard3_replica2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_A23057AEB5080A8B-001/tempDir-001/node1/routeFieldColl_shard3_replica2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_A23057AEB5080A8B-001/tempDir-001/node1/./routeFieldColl_shard3_replica2/data/]
[junit4] 2> 1753340 INFO (qtp1239476393-13379) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.c.CoreContainer Creating SolrCore 'routeFieldColl_shard2_replica1' using configuration from collection routeFieldColl
[junit4] 2> 1753340 INFO (qtp1239476393-13379) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.c.SolrCore [[routeFieldColl_shard2_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_A23057AEB5080A8B-001/tempDir-001/node4/routeFieldColl_shard2_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_A23057AEB5080A8B-001/tempDir-001/node4/./routeFieldColl_shard2_replica1/data/]
[junit4] 2> 1753342 INFO (qtp967218453-13366) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.c.CoreContainer Creating SolrCore 'routeFieldColl_shard3_replica1' using configuration from collection routeFieldColl
[junit4] 2> 1753342 INFO (qtp967218453-13366) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.c.SolrCore [[routeFieldColl_shard3_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_A23057AEB5080A8B-001/tempDir-001/node3/routeFieldColl_shard3_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_A23057AEB5080A8B-001/tempDir-001/node3/./routeFieldColl_shard3_replica1/data/]
[junit4] 2> 1753349 INFO (qtp967218453-13368) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.s.IndexSchema [routeFieldColl_shard1_replica1] Schema name=cloud-dynamic
[junit4] 2> 1753350 INFO (qtp1239476393-13377) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.c.CoreContainer Creating SolrCore 'routeFieldColl_shard4_replica1' using configuration from collection routeFieldColl
[junit4] 2> 1753350 INFO (qtp1239476393-13377) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.c.SolrCore [[routeFieldColl_shard4_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_A23057AEB5080A8B-001/tempDir-001/node4/routeFieldColl_shard4_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_A23057AEB5080A8B-001/tempDir-001/node4/./routeFieldColl_shard4_replica1/data/]
[junit4] 2> 1753360 WARN (qtp582900176-13355) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.s.IndexSchema [routeFieldColl_shard2_replica2] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1753361 INFO (qtp582900176-13355) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 1753362 INFO (qtp582900176-13355) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.c.CoreContainer Creating SolrCore 'routeFieldColl_shard2_replica2' using configuration from collection routeFieldColl
[junit4] 2> 1753363 INFO (qtp582900176-13355) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.c.SolrCore [[routeFieldColl_shard2_replica2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_A23057AEB5080A8B-001/tempDir-001/node2/routeFieldColl_shard2_replica2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_A23057AEB5080A8B-001/tempDir-001/node2/./routeFieldColl_shard2_replica2/data/]
[junit4] 2> 1753377 WARN (qtp967218453-13368) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.s.IndexSchema [routeFieldColl_shard1_replica1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1753378 INFO (qtp967218453-13368) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 1753379 INFO (qtp967218453-13368) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'routeFieldColl_shard1_replica1' using configuration from collection routeFieldColl
[junit4] 2> 1753380 INFO (qtp967218453-13368) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.c.SolrCore [[routeFieldColl_shard1_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_A23057AEB5080A8B-001/tempDir-001/node3/routeFieldColl_shard1_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_A23057AEB5080A8B-001/tempDir-001/node3/./routeFieldColl_shard1_replica1/data/]
[junit4] 2> 1753499 INFO (qtp1239476393-13379) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1753499 INFO (qtp1239476393-13379) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1753501 INFO (qtp1239476393-13379) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1753501 INFO (qtp1239476393-13379) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1753517 INFO (qtp1958637594-13385) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1753517 INFO (qtp1958637594-13385) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1753519 INFO (qtp1958637594-13385) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1753519 INFO (qtp1958637594-13385) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1753522 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1753522 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1753524 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1753524 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1753534 INFO (qtp967218453-13368) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1753534 INFO (qtp967218453-13368) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1753538 INFO (qtp582900176-13357) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1753538 INFO (qtp582900176-13357) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1753542 INFO (qtp967218453-13366) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1753543 INFO (qtp967218453-13366) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1753546 INFO (qtp1239476393-13379) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@63a3db9[routeFieldColl_shard2_replica1] main]
[junit4] 2> 1753549 INFO (qtp1239476393-13379) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 1753549 INFO (qtp1239476393-13379) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 1753549 INFO (qtp1239476393-13377) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1753550 INFO (qtp1239476393-13379) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1753550 INFO (searcherExecutor-5540-thread-1-processing-n:127.0.0.1:52874_solr x:routeFieldColl_shard2_replica1 s:shard2 c:routeFieldColl) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.c.SolrCore [routeFieldColl_shard2_replica1] Registered new searcher Searcher@63a3db9[routeFieldColl_shard2_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1753550 INFO (qtp967218453-13366) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1753550 INFO (qtp967218453-13366) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1753550 INFO (qtp1239476393-13377) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1753550 INFO (qtp582900176-13357) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1753550 INFO (qtp582900176-13357) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1753550 INFO (qtp1239476393-13379) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559083755669291008
[junit4] 2> 1753551 INFO (qtp1239476393-13377) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1753552 INFO (qtp1239476393-13377) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1753552 INFO (qtp967218453-13368) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1753552 INFO (qtp967218453-13368) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1753557 INFO (qtp1958637594-13385) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@73b63cbe[routeFieldColl_shard3_replica2] main]
[junit4] 2> 1753562 INFO (qtp1958637594-13385) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 1753563 INFO (qtp1958637594-13385) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 1753563 INFO (qtp1958637594-13385) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1753563 INFO (qtp1239476393-13379) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoutin=9999ms
[junit4] 2> 1753563 INFO (searcherExecutor-5539-thread-1-processing-n:127.0.0.1:52875_solr x:routeFieldColl_shard3_replica2 s:shard3 c:routeFieldColl) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.c.SolrCore [routeFieldColl_shard3_replica2] Registered new searcher Searcher@73b63cbe[routeFieldColl_shard3_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1753563 INFO (qtp1958637594-13385) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559083755682922496
[junit4] 2> 1753567 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@5efed849[routeFieldColl_shard1_replica2] main]
[junit4] 2> 1753570 INFO (qtp582900176-13355) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1753570 INFO (qtp582900176-13355) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1753570 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 1753570 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 1753571 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1753571 INFO (searcherExecutor-5538-thread-1-processing-n:127.0.0.1:52875_solr x:routeFieldColl_shard1_replica2 s:shard1 c:routeFieldColl) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.c.SolrCore [routeFieldColl_shard1_replica2] Registered new searcher Searcher@5efed849[routeFieldColl_shard1_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1753571 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559083755691311104
[junit4] 2> 1753574 INFO (qtp582900176-13355) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1753574 INFO (qtp582900176-13355) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1753575 INFO (qtp1239476393-13377) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@30d1dc6f[routeFieldColl_shard4_replica1] main]
[junit4] 2> 1753575 INFO (qtp1958637594-13385) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard3: total=2 found=1 timeoutin=9999ms
[junit4] 2> 1753577 INFO (qtp1239476393-13377) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 1753578 INFO (qtp1239476393-13377) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 1753578 INFO (qtp1239476393-13377) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1753578 INFO (searcherExecutor-5542-thread-1-processing-n:127.0.0.1:52874_solr x:routeFieldColl_shard4_replica1 s:shard4 c:routeFieldColl) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.c.SolrCore [routeFieldColl_shard4_replica1] Registered new searcher Searcher@30d1dc6f[routeFieldColl_shard4_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1753578 INFO (qtp1239476393-13377) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559083755698651136
[junit4] 2> 1753580 INFO (qtp967218453-13366) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4b82a11b[routeFieldColl_shard3_replica1] main]
[junit4] 2> 1753583 INFO (qtp967218453-13366) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 1753583 INFO (qtp582900176-13357) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@77712410[routeFieldColl_shard4_replica2] main]
[junit4] 2> 1753584 INFO (qtp967218453-13366) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 1753584 INFO (qtp967218453-13366) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1753584 INFO (searcherExecutor-5541-thread-1-processing-n:127.0.0.1:52873_solr x:routeFieldColl_shard3_replica1 s:shard3 c:routeFieldColl) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.c.SolrCore [routeFieldColl_shard3_replica1] Registered new searcher Searcher@4b82a11b[routeFieldColl_shard3_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1753584 INFO (qtp967218453-13366) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559083755704942592
[junit4] 2> 1753586 INFO (qtp582900176-13357) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 1753587 INFO (qtp582900176-13357) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 1753587 INFO (qtp582900176-13357) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1753587 INFO (searcherExecutor-5537-thread-1-processing-n:127.0.0.1:52872_solr x:routeFieldColl_shard4_replica2 s:shard4 c:routeFieldColl) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica2] o.a.s.c.SolrCore [routeFieldColl_shard4_replica2] Registered new searcher Searcher@77712410[routeFieldColl_shard4_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1753588 INFO (qtp582900176-13357) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559083755709136896
[junit4] 2> 1753589 INFO (qtp1239476393-13377) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard4: total=2 found=1 timeoutin=9999ms
[junit4] 2> 1753593 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9998ms
[junit4] 2> 1753594 INFO (qtp967218453-13368) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@f1b1294[routeFieldColl_shard1_replica1] main]
[junit4] 2> 1753596 INFO (qtp967218453-13368) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 1753596 INFO (qtp582900176-13355) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@3bc1cf54[routeFieldColl_shard2_replica2] main]
[junit4] 2> 1753597 INFO (qtp967218453-13368) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 1753597 INFO (qtp967218453-13368) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1753597 INFO (qtp967218453-13368) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559083755718574080
[junit4] 2> 1753597 INFO (searcherExecutor-5544-thread-1-processing-n:127.0.0.1:52873_solr x:routeFieldColl_shard1_replica1 s:shard1 c:routeFieldColl) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.c.SolrCore [routeFieldColl_shard1_replica1] Registered new searcher Searcher@f1b1294[routeFieldColl_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1753598 INFO (qtp582900176-13355) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 1753598 INFO (qtp582900176-13355) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 1753598 INFO (qtp582900176-13355) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1753598 INFO (qtp582900176-13355) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559083755719622656
[junit4] 2> 1753598 INFO (searcherExecutor-5543-thread-1-processing-n:127.0.0.1:52872_solr x:routeFieldColl_shard2_replica2 s:shard2 c:routeFieldColl) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.c.SolrCore [routeFieldColl_shard2_replica2] Registered new searcher Searcher@3bc1cf54[routeFieldColl_shard2_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1753699 INFO (zkCallback-2853-thread-1-processing-n:127.0.0.1:52873_solr) [n:127.0.0.1:52873_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routeFieldColl/state.json] for collection [routeFieldColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1753699 INFO (zkCallback-2855-thread-1-processing-n:127.0.0.1:52874_solr) [n:127.0.0.1:52874_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routeFieldColl/state.json] for collection [routeFieldColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1753700 INFO (zkCallback-2852-thread-3-processing-n:127.0.0.1:52875_solr) [n:127.0.0.1:52875_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routeFieldColl/state.json] for collection [routeFieldColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1753699 INFO (zkCallback-2854-thread-1-processing-n:127.0.0.1:52872_solr) [n:127.0.0.1:52872_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routeFieldColl/state.json] for collection [routeFieldColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1754075 INFO (qtp1239476393-13379) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1754075 INFO (qtp1239476393-13379) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1754075 INFO (qtp1239476393-13379) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:52874/solr/routeFieldColl_shard2_replica1/
[junit4] 2> 1754075 INFO (qtp1239476393-13379) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.u.PeerSync PeerSync: core=routeFieldColl_shard2_replica1 url=http://127.0.0.1:52874/solr START replicas=[http://127.0.0.1:52872/solr/routeFieldColl_shard2_replica2/] nUpdates=100
[junit4] 2> 1754076 INFO (qtp1239476393-13379) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.u.PeerSync PeerSync: core=routeFieldColl_shard2_replica1 url=http://127.0.0.1:52874/solr DONE. We have no versions. sync failed.
[junit4] 2> 1754076 INFO (qtp1958637594-13385) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1754076 INFO (qtp1958637594-13385) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1754077 INFO (qtp1958637594-13385) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:52875/solr/routeFieldColl_shard3_replica2/
[junit4] 2> 1754077 INFO (qtp1958637594-13385) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.u.PeerSync PeerSync: core=routeFieldColl_shard3_replica2 url=http://127.0.0.1:52875/solr START replicas=[http://127.0.0.1:52873/solr/routeFieldColl_shard3_replica1/] nUpdates=100
[junit4] 2> 1754077 INFO (qtp1958637594-13385) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.u.PeerSync PeerSync: core=routeFieldColl_shard3_replica2 url=http://127.0.0.1:52875/solr DONE. We have no versions. sync failed.
[junit4] 2> 1754078 INFO (qtp582900176-13360) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard2 r:core_node6 x:routeFieldColl_shard2_replica2] o.a.s.c.S.Request [routeFieldColl_shard2_replica2] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1754079 INFO (qtp1239476393-13379) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 1754079 INFO (qtp1239476393-13379) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 1754080 INFO (qtp967218453-13370) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard3 r:core_node3 x:routeFieldColl_shard3_replica1] o.a.s.c.S.Request [routeFieldColl_shard3_replica1] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1754080 INFO (qtp1958637594-13385) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 1754080 INFO (qtp1958637594-13385) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 1754087 INFO (qtp1239476393-13379) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:52874/solr/routeFieldColl_shard2_replica1/ shard2
[junit4] 2> 1754088 INFO (qtp1958637594-13385) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:52875/solr/routeFieldColl_shard3_replica2/ shard3
[junit4] 2> 1754090 INFO (qtp1239476393-13377) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1754090 INFO (qtp1239476393-13377) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1754090 INFO (qtp1239476393-13377) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:52874/solr/routeFieldColl_shard4_replica1/
[junit4] 2> 1754091 INFO (qtp1239476393-13377) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.u.PeerSync PeerSync: core=routeFieldColl_shard4_replica1 url=http://127.0.0.1:52874/solr START replicas=[http://127.0.0.1:52872/solr/routeFieldColl_shard4_replica2/] nUpdates=100
[junit4] 2> 1754091 INFO (qtp1239476393-13377) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.u.PeerSync PeerSync: core=routeFieldColl_shard4_replica1 url=http://127.0.0.1:52874/solr DONE. We have no versions. sync failed.
[junit4] 2> 1754093 INFO (qtp582900176-13361) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard4 r:core_node7 x:routeFieldColl_shard4_replica2] o.a.s.c.S.Request [routeFieldColl_shard4_replica2] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 1754093 INFO (qtp1239476393-13377) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 1754093 INFO (qtp1239476393-13377) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 1754094 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1754094 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1754094 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:52875/solr/routeFieldColl_shard1_replica2/
[junit4] 2> 1754095 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.u.PeerSync PeerSync: core=routeFieldColl_shard1_replica2 url=http://127.0.0.1:52875/solr START replicas=[http://127.0.0.1:52873/solr/routeFieldColl_shard1_replica1/] nUpdates=100
[junit4] 2> 1754095 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.u.PeerSync PeerSync: core=routeFieldColl_shard1_replica2 url=http://127.0.0.1:52875/solr DONE. We have no versions. sync failed.
[junit4] 2> 1754095 INFO (qtp967218453-13371) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard1 r:core_node4 x:routeFieldColl_shard1_replica1] o.a.s.c.S.Request [routeFieldColl_shard1_replica1] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1754096 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 1754096 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 1754102 INFO (qtp1239476393-13377) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:52874/solr/routeFieldColl_shard4_replica1/ shard4
[junit4] 2> 1754103 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:52875/solr/routeFieldColl_shard1_replica2/ shard1
[junit4] 2> 1754218 INFO (zkCallback-2852-thread-2-processing-n:127.0.0.1:52875_solr) [n:127.0.0.1:52875_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routeFieldColl/state.json] for collection [routeFieldColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1754218 INFO (zkCallback-2855-thread-1-processing-n:127.0.0.1:52874_solr) [n:127.0.0.1:52874_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routeFieldColl/state.json] for collection [routeFieldColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1754218 INFO (zkCallback-2853-thread-1-processing-n:127.0.0.1:52873_solr) [n:127.0.0.1:52873_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routeFieldColl/state.json] for collection [routeFieldColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1754218 INFO (zkCallback-2854-thread-1-processing-n:127.0.0.1:52872_solr) [n:127.0.0.1:52872_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routeFieldColl/state.json] for collection [routeFieldColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1754261 INFO (qtp1239476393-13379) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1754262 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1754262 INFO (qtp1958637594-13385) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1754261 INFO (qtp1239476393-13377) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1754266 INFO (qtp1239476393-13379) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=routeFieldColl_shard2_replica1&action=CREATE&numShards=4&collection=routeFieldColl&shard=shard2&wt=javabin&version=2} status=0 QTime=2126
[junit4] 2> 1754266 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=routeFieldColl_shard1_replica2&action=CREATE&numShards=4&collection=routeFieldColl&shard=shard1&wt=javabin&version=2} status=0 QTime=2127
[junit4] 2> 1754267 INFO (qtp1239476393-13377) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=routeFieldColl_shard4_replica1&action=CREATE&numShards=4&collection=routeFieldColl&shard=shard4&wt=javabin&version=2} status=0 QTime=2128
[junit4] 2> 1754267 INFO (qtp1958637594-13385) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=routeFieldColl_shard3_replica2&action=CREATE&numShards=4&collection=routeFieldColl&shard=shard3&wt=javabin&version=2} status=0 QTime=2125
[junit4] 2> 1754387 INFO (zkCallback-2852-thread-1-processing-n:127.0.0.1:52875_solr) [n:127.0.0.1:52875_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routeFieldColl/state.json] for collection [routeFieldColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1754387 INFO (zkCallback-2855-thread-1-processing-n:127.0.0.1:52874_solr) [n:127.0.0.1:52874_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routeFieldColl/state.json] for collection [routeFieldColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1754387 INFO (zkCallback-2853-thread-1-processing-n:127.0.0.1:52873_solr) [n:127.0.0.1:52873_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routeFieldColl/state.json] for collection [routeFieldColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1754387 INFO (zkCallback-2854-thread-1-processing-n:127.0.0.1:52872_solr) [n:127.0.0.1:52872_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routeFieldColl/state.json] for collection [routeFieldColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1754607 INFO (qtp582900176-13357) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=routeFieldColl_shard4_replica2&action=CREATE&numShards=4&collection=routeFieldColl&shard=shard4&wt=javabin&version=2} status=0 QTime=2463
[junit4] 2> 1754607 INFO (qtp967218453-13366) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=routeFieldColl_shard3_replica1&action=CREATE&numShards=4&collection=routeFieldColl&shard=shard3&wt=javabin&version=2} status=0 QTime=2468
[junit4] 2> 1754612 INFO (qtp967218453-13368) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=routeFieldColl_shard1_replica1&action=CREATE&numShards=4&collection=routeFieldColl&shard=shard1&wt=javabin&version=2} status=0 QTime=2472
[junit4] 2> 1754612 INFO (qtp582900176-13355) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=routeFieldColl_shard2_replica2&action=CREATE&numShards=4&collection=routeFieldColl&shard=shard2&wt=javabin&version=2} status=0 QTime=2472
[junit4] 2> 1754618 INFO (qtp1239476393-13375) [n:127.0.0.1:52874_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1754733 INFO (zkCallback-2855-thread-1-processing-n:127.0.0.1:52874_solr) [n:127.0.0.1:52874_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routeFieldColl/state.json] for collection [routeFieldColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1754733 INFO (zkCallback-2853-thread-1-processing-n:127.0.0.1:52873_solr) [n:127.0.0.1:52873_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routeFieldColl/state.json] for collection [routeFieldColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1754733 INFO (zkCallback-2852-thread-3-processing-n:127.0.0.1:52875_solr) [n:127.0.0.1:52875_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routeFieldColl/state.json] for collection [routeFieldColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1754733 INFO (zkCallback-2854-thread-1-processing-n:127.0.0.1:52872_solr) [n:127.0.0.1:52872_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routeFieldColl/state.json] for collection [routeFieldColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1755619 INFO (qtp1239476393-13375) [n:127.0.0.1:52874_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=conf&maxShardsPerNode=3&router.field=shard_s&name=routeFieldColl&action=CREATE&numShards=4&wt=javabin&version=2} status=0 QTime=3609
[junit4] 2> 1755631 INFO (qtp967218453-13367) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard3 r:core_node3 x:routeFieldColl_shard3_replica1] o.a.s.u.p.LogUpdateProcessorFactory [routeFieldColl_shard3_replica1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52875/solr/routeFieldColl_shard3_replica2/&wt=javabin&version=2}{add=[6 (1559083757844037632), 7 (1559083757848231936)]} 0 2
[junit4] 2> 1755670 INFO (qtp1958637594-13386) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 r:core_node8 x:routeFieldColl_shard3_replica2] o.a.s.u.p.LogUpdateProcessorFactory [routeFieldColl_shard3_replica2] webapp=/solr path=/update params={_stateVer_=routeFieldColl:5&wt=javabin&version=2}{add=[6 (1559083757844037632), 7 (1559083757848231936)]} 0 46
[junit4] 2> 1755670 INFO (qtp967218453-13369) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard1 r:core_node4 x:routeFieldColl_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [routeFieldColl_shard1_replica1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52875/solr/routeFieldColl_shard1_replica2/&wt=javabin&version=2}{add=[8 (1559083757844037632)]} 0 40
[junit4] 2> 1755671 INFO (qtp1958637594-13388) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 r:core_node5 x:routeFieldColl_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [routeFieldColl_shard1_replica2] webapp=/solr path=/update params={_stateVer_=routeFieldColl:5&wt=javabin&version=2}{add=[8 (1559083757844037632)]} 0 47
[junit4] 2> 1755672 INFO (qtp967218453-13370) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard1 r:core_node4 x:routeFieldColl_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1755672 INFO (qtp967218453-13370) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard1 r:core_node4 x:routeFieldColl_shard1_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@49578ee
[junit4] 2> 1755673 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 r:core_node5 x:routeFieldColl_shard1_replica2] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1755673 INFO (qtp1958637594-13384) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard1 r:core_node5 x:routeFieldColl_shard1_replica2] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@2320d084
[junit4] 2> 1755673 INFO (qtp582900176-13362) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard2 r:core_node6 x:routeFieldColl_shard2_replica2] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1755673 INFO (qtp582900176-13362) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard2 r:core_node6 x:routeFieldColl_shard2_replica2] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 1755675 INFO (qtp1239476393-13380) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 r:core_node2 x:routeFieldColl_shard2_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1755675 INFO (qtp1239476393-13380) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 r:core_node2 x:routeFieldColl_shard2_replica1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 1755675 INFO (qtp582900176-13362) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard2 r:core_node6 x:routeFieldColl_shard2_replica2] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1755675 INFO (qtp582900176-13362) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard2 r:core_node6 x:routeFieldColl_shard2_replica2] o.a.s.u.p.LogUpdateProcessorFactory [routeFieldColl_shard2_replica2] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:52875/solr/routeFieldColl_shard3_replica2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 1
[junit4] 2> 1755676 INFO (qtp967218453-13371) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard3 r:core_node3 x:routeFieldColl_shard3_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1755676 INFO (qtp967218453-13371) [n:127.0.0.1:52873_solr c:routeFieldColl s:shard3 r:core_node3 x:routeFieldColl_shard3_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@4124fbb2
[junit4] 2> 1755677 INFO (qtp1239476393-13380) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 r:core_node2 x:routeFieldColl_shard2_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1755677 INFO (qtp1239476393-13380) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard2 r:core_node2 x:routeFieldColl_shard2_replica1] o.a.s.u.p.LogUpdateProcessorFactory [routeFieldColl_shard2_replica1] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:52875/solr/routeFieldColl_shard3_replica2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 2
[junit4] 2> 1755677 INFO (qtp1239476393-13381) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 r:core_node1 x:routeFieldColl_shard4_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1755678 INFO (qtp1239476393-13381) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 r:core_node1 x:routeFieldColl_shard4_replica1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 1755679 INFO (qtp1239476393-13381) [n:127.0.0.1:52874_solr c:routeFieldColl s:shard4 r:core_node1 x:routeFieldColl_shard4_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1755679 INFO (qtp582900176-13358) [n:127.0.0.1:52872_solr c:routeFieldColl s:shard4 r:core_node7 x:routeFieldColl_shard4_replica2] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1755679 INFO (qtp1958637594-13387) [n:127.0.0.1:52875_solr c:routeFieldColl s:shard3 r:core_node8 x:routeFieldColl_shard3_replica2] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSe
[...truncated too long message...]
junit4] > at org.apache.solr.cloud.PeerSyncReplicationTest.test(PeerSyncReplicationTest.java:138)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 3358094 INFO (SUITE-PeerSyncReplicationTest-seed#[A23057AEB5080A8B]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_A23057AEB5080A8B-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {other_tl1=PostingsFormat(name=Direct), range_facet_l_dv=PostingsFormat(name=LuceneVarGapFixedInterval), rnd_s=PostingsFormat(name=Direct), multiDefault=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), a_t=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), intDefault=PostingsFormat(name=Direct), a_i1=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), id=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_i_dv=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), text=Lucene50(blocksize=128), range_facet_l=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), timestamp=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128)))}, docValues:{range_facet_l_dv=DocValuesFormat(name=Lucene70), _version_=DocValuesFormat(name=Lucene70), range_facet_i_dv=DocValuesFormat(name=Memory), intDvoDefault=DocValuesFormat(name=Lucene70), timestamp=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=789, maxMBSortInHeap=6.54940252975365, sim=RandomSimilarity(queryNorm=false): {}, locale=ru-RU, timezone=Asia/Rangoon
[junit4] 2> NOTE: Mac OS X 10.11.6 x86_64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=3,threads=1,free=273524096,total=515244032
[junit4] 2> NOTE: All tests run in this JVM: [TestUnifiedSolrHighlighter, StatsComponentTest, DistributedSuggestComponentTest, SignatureUpdateProcessorFactoryTest, UpdateRequestProcessorFactoryTest, ChaosMonkeySafeLeaderTest, ParsingFieldUpdateProcessorsTest, CoreMergeIndexesAdminHandlerTest, TestBadConfig, TestCrossCoreJoin, DateMathParserTest, TestStandardQParsers, VMParamsZkACLAndCredentialsProvidersTest, TestExactStatsCache, JsonLoaderTest, TestIndexingPerformance, SharedFSAutoReplicaFailoverTest, ScriptEngineTest, TestFastWriter, TestConfigSets, TestStressInPlaceUpdates, TestClusterStateMutator, SortByFunctionTest, TestManagedResourceStorage, HttpPartitionTest, ZkSolrClientTest, TestShortCircuitedRequests, TestSolrConfigHandlerCloud, CoreAdminHandlerTest, TestConfigSetImmutable, SuggesterTSTTest, FieldAnalysisRequestHandlerTest, CSVRequestHandlerTest, OpenCloseCoreStressTest, TestGeoJSONResponseWriter, TestExceedMaxTermLength, BasicAuthIntegrationTest, SolrGangliaReporterTest, TestPKIAuthenticationPlugin, TestReloadAndDeleteDocs, TestConfigsApi, BlockCacheTest, TestLeaderElectionZkExpiry, LeaderFailoverAfterPartitionTest, RuleEngineTest, CdcrReplicationDistributedZkTest, TestSchemaManager, BlobRepositoryCloudTest, TestSolrCoreProperties, SolrCoreCheckLockOnStartupTest, TestUtils, SubstringBytesRefFilterTest, TestSQLHandler, ConjunctionSolrSpellCheckerTest, InfixSuggestersTest, TestSubQueryTransformerDistrib, TestCollapseQParserPlugin, TestOrdValues, CacheHeaderTest, TestCustomStream, TestSolrQueryParserResource, TestSha256AuthenticationProvider, SaslZkACLProviderTest, TestComplexPhraseQParserPlugin, TestSerializedLuceneMatchVersion, SpellingQueryConverterTest, TestManagedStopFilterFactory, TestFieldTypeCollectionResource, TestSolrDeletionPolicy2, TestRawResponseWriter, TestTestInjection, TestDistribIDF, TestDFRSimilarityFactory, SolrCmdDistributorTest, DeleteReplicaTest, TestMiniSolrCloudCluster, TestCloudInspectUtil, AutoCommitTest, TestComponentsName, LeaderInitiatedRecoveryOnCommitTest, CollectionsAPIAsyncDistributedZkTest, TestAtomicUpdateErrorCases, PKIAuthenticationIntegrationTest, TestJoin, HLLUtilTest, TestCoreDiscovery, TestUninvertingReader, RankQueryTest, TestManagedSchema, RAMDirectoryFactoryTest, TestSolrCloudWithDelegationTokens, TestScoreJoinQPScore, NoCacheHeaderTest, TestMiniSolrCloudClusterSSL, TestLMDirichletSimilarityFactory, TestDeleteCollectionOnDownNodes, TestBulkSchemaAPI, CoreAdminCreateDiscoverTest, BasicDistributedZk2Test, RegexBoostProcessorTest, TestSolr4Spatial, CachingDirectoryFactoryTest, MigrateRouteKeyTest, ClassificationUpdateProcessorFactoryTest, TestLuceneMatchVersion, TestJsonFacets, CdcrRequestHandlerTest, FieldMutatingUpdateProcessorTest, BadIndexSchemaTest, TestUseDocValuesAsStored, TermVectorComponentDistributedTest, DefaultValueUpdateProcessorTest, ResponseLogComponentTest, TestDocSet, SuggesterTest, DirectoryFactoryTest, BigEndianAscendingWordDeserializerTest, GraphQueryTest, DirectUpdateHandlerOptimizeTest, TestJettySolrRunner, ReplicationFactorTest, CursorMarkTest, HighlighterConfigTest, TestMaxScoreQueryParser, DocValuesMissingTest, DistributedFacetPivotSmallTest, TestZkChroot, SegmentsInfoRequestHandlerTest, TestSchemaSimilarityResource, SolrMetricReporterTest, TestTrieFacet, DistributedIntervalFacetingTest, TestReload, TestIndexSearcher, DeleteStatusTest, TestRecoveryHdfs, TestImplicitCoreProperties, TestXmlQParserPlugin, TestValueSourceCache, TestHdfsBackupRestoreCore, DistributedFacetExistsSmallTest, TestSolrCoreSnapshots, TestConfigSetsAPI, CustomCollectionTest, TestSchemaNameResource, LukeRequestHandlerTest, ShardRoutingCustomTest, TestIntervalFaceting, IndexSchemaTest, ShowFileRequestHandlerTest, FileBasedSpellCheckerTest, MinimalSchemaTest, BitVectorTest, TestRTimerTree, AnalysisErrorHandlingTest, TestSolrQueryParser, TestCopyFieldCollectionResource, OverseerStatusTest, OverriddenZkACLAndCredentialsProvidersTest, TestQueryTypes, TestSSLRandomization, HdfsBasicDistributedZkTest, TestLMJelinekMercerSimilarityFactory, TestRandomFaceting, UpdateParamsTest, TestAnalyzedSuggestions, BlockJoinFacetRandomTest, TestSystemIdResolver, SolrCloudExampleTest, TestWriterPerf, TlogReplayBufferedWhileIndexingTest, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest, LeaderElectionIntegrationTest, BasicZkTest, FullSolrCloudDistribCmdsTest, LeaderElectionTest, TestDistributedGrouping, DistributedSpellCheckComponentTest, ZkControllerTest, HardAutoCommitTest, TestRangeQuery, CurrencyFieldOpenExchangeTest, SolrIndexSplitterTest, SpellCheckCollatorTest, SuggesterFSTTest, SolrRequestParserTest, TestFoldingMultitermQuery, DocValuesTest, StatelessScriptUpdateProcessorFactoryTest, SolrInfoMBeanTest, IndexBasedSpellCheckerTest, StandardRequestHandlerTest, DocumentAnalysisRequestHandlerTest, TestOmitPositions, XmlUpdateRequestHandlerTest, RequestHandlersTest, PathHierarchyTokenizerFactoryTest, MoreLikeThisHandlerTest, LoggingHandlerTest, TestJmxIntegration, TestPartialUpdateDeduplication, TestLFUCache, TestMergePolicyConfig, MultiTermTest, TestSearchPerf, TestConfig, TestFuzzyAnalyzedSuggestions, ExternalFileFieldSortTest, TestPostingsSolrHighlighter, DirectSolrConnectionTest, NotRequiredUniqueKeyTest, SpellPossibilityIteratorTest, TestCharFilters, TestCodecSupport, SynonymTokenizerTest, TestXIncludeConfig, EchoParamsTest, TestSweetSpotSimilarityFactory, TestIBSimilarityFactory, TestBM25SimilarityFactory, TestJmxMonitoredMap, ResourceLoaderTest, TestFastOutputStream, URLClassifyProcessorTest, PluginInfoTest, PreAnalyzedFieldTest, PrimUtilsTest, TestSuggestSpellingConverter, DateFieldTest, ClusterStateTest, TestSolrJ, TestLRUCache, SliceStateTest, UUIDFieldTest, FileUtilsTest, CircularListTest, TestRTGBase, SolrTestCaseJ4Test, TestCursorMarkWithoutUniqueKey, TestDistributedMissingSort, TestSimpleTrackingShardHandler, AssignTest, BaseCdcrDistributedZkTest, CleanupOldIndexTest, CollectionReloadTest, CollectionTooManyReplicasTest, ConcurrentDeleteAndCreateCollectionTest, ConnectionManagerTest, CreateCollectionCleanupTest, DeleteShardTest, DistribJoinFromCollectionTest, DistributedQueueTest, ForceLeaderTest, PeerSyncReplicationTest]
[junit4] Completed [510/694 (2!)] on J1 in 219.07s, 1 test, 1 failure <<< FAILURES!
[...truncated 62792 lines...]
-documentation-lint:
[jtidy] Checking for broken html (such as invalid tags)...
[delete] Deleting directory /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/jtidy_tmp
[echo] Checking for broken links...
[exec]
[exec] Crawl/parse...
[exec]
[exec] Verify...
[echo] Checking for malformed docs...
[exec]
[exec] /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/docs/solr-solrj/overview-summary.html
[exec] missing: org.apache.solr.client.solrj.io.eval
[exec]
[exec] Missing javadocs were found!
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/build.xml:775: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/build.xml:101: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build.xml:654: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build.xml:670: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/common-build.xml:2493: exec returned: 1
Total time: 119 minutes 25 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3825
- Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3825/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC
1 tests failed.
FAILED: org.apache.solr.cloud.CustomCollectionTest.testRouteFieldForHashRouter
Error Message:
Collection not found: routeFieldColl
Stack Trace:
org.apache.solr.common.SolrException: Collection not found: routeFieldColl
at __randomizedtesting.SeedInfo.seed([167C8A2C2AD6C4E3:BE4A14F1B5B72FB9]:0)
at org.apache.solr.client.solrj.impl.CloudSolrClient.getCollectionNames(CloudSolrClient.java:1376)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1072)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1042)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:232)
at org.apache.solr.cloud.CustomCollectionTest.testRouteFieldForHashRouter(CustomCollectionTest.java:166)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 12332 lines...]
[junit4] Suite: org.apache.solr.cloud.CustomCollectionTest
[junit4] 2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/init-core-data-001
[junit4] 2> 2361042 INFO (SUITE-CustomCollectionTest-seed#[167C8A2C2AD6C4E3]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields
[junit4] 2> 2361043 INFO (SUITE-CustomCollectionTest-seed#[167C8A2C2AD6C4E3]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN) w/ MAC_OS_X supressed clientAuth
[junit4] 2> 2361044 INFO (SUITE-CustomCollectionTest-seed#[167C8A2C2AD6C4E3]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001
[junit4] 2> 2361044 INFO (SUITE-CustomCollectionTest-seed#[167C8A2C2AD6C4E3]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2361045 INFO (Thread-3217) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2361045 INFO (Thread-3217) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2361147 INFO (SUITE-CustomCollectionTest-seed#[167C8A2C2AD6C4E3]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:63796
[junit4] 2> 2361165 INFO (jetty-launcher-2359-thread-1) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2361165 INFO (jetty-launcher-2359-thread-2) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2361165 INFO (jetty-launcher-2359-thread-3) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2361166 INFO (jetty-launcher-2359-thread-4) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2361169 INFO (jetty-launcher-2359-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@42f9fddd{/solr,null,AVAILABLE}
[junit4] 2> 2361170 INFO (jetty-launcher-2359-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@12a19f46{/solr,null,AVAILABLE}
[junit4] 2> 2361171 INFO (jetty-launcher-2359-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6bc8c0a9{HTTP/1.1,[http/1.1]}{127.0.0.1:63798}
[junit4] 2> 2361171 INFO (jetty-launcher-2359-thread-2) [ ] o.e.j.s.Server Started @2366429ms
[junit4] 2> 2361171 INFO (jetty-launcher-2359-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=63798}
[junit4] 2> 2361171 ERROR (jetty-launcher-2359-thread-2) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2361172 INFO (jetty-launcher-2359-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 2361172 INFO (jetty-launcher-2359-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7d606966{HTTP/1.1,[http/1.1]}{127.0.0.1:63799}
[junit4] 2> 2361172 INFO (jetty-launcher-2359-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2361172 INFO (jetty-launcher-2359-thread-4) [ ] o.e.j.s.Server Started @2366430ms
[junit4] 2> 2361172 INFO (jetty-launcher-2359-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2361172 INFO (jetty-launcher-2359-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=63799}
[junit4] 2> 2361172 INFO (jetty-launcher-2359-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-11T11:33:57.388Z
[junit4] 2> 2361172 ERROR (jetty-launcher-2359-thread-4) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2361172 INFO (jetty-launcher-2359-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 2361172 INFO (jetty-launcher-2359-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2361173 INFO (jetty-launcher-2359-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2361173 INFO (jetty-launcher-2359-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-11T11:33:57.389Z
[junit4] 2> 2361173 INFO (jetty-launcher-2359-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4191455{/solr,null,AVAILABLE}
[junit4] 2> 2361174 INFO (jetty-launcher-2359-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@23484329{HTTP/1.1,[http/1.1]}{127.0.0.1:63800}
[junit4] 2> 2361174 INFO (jetty-launcher-2359-thread-3) [ ] o.e.j.s.Server Started @2366432ms
[junit4] 2> 2361174 INFO (jetty-launcher-2359-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=63800}
[junit4] 2> 2361174 ERROR (jetty-launcher-2359-thread-3) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2361175 INFO (jetty-launcher-2359-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 2361175 INFO (jetty-launcher-2359-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2361175 INFO (jetty-launcher-2359-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2361175 INFO (jetty-launcher-2359-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-11T11:33:57.391Z
[junit4] 2> 2361176 INFO (jetty-launcher-2359-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@540c4a5d{/solr,null,AVAILABLE}
[junit4] 2> 2361176 INFO (jetty-launcher-2359-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2361177 INFO (jetty-launcher-2359-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2361178 INFO (jetty-launcher-2359-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4d205557{HTTP/1.1,[http/1.1]}{127.0.0.1:63803}
[junit4] 2> 2361178 INFO (jetty-launcher-2359-thread-1) [ ] o.e.j.s.Server Started @2366436ms
[junit4] 2> 2361178 INFO (jetty-launcher-2359-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=63803}
[junit4] 2> 2361179 ERROR (jetty-launcher-2359-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2361179 INFO (jetty-launcher-2359-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 2361179 INFO (jetty-launcher-2359-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2361179 INFO (jetty-launcher-2359-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2361179 INFO (jetty-launcher-2359-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-11T11:33:57.395Z
[junit4] 2> 2361182 INFO (jetty-launcher-2359-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2361187 INFO (jetty-launcher-2359-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2361192 INFO (jetty-launcher-2359-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:63796/solr
[junit4] 2> 2361196 INFO (jetty-launcher-2359-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:63796/solr
[junit4] 2> 2361200 INFO (jetty-launcher-2359-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:63796/solr
[junit4] 2> 2361203 INFO (jetty-launcher-2359-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:63796/solr
[junit4] 2> 2361236 INFO (jetty-launcher-2359-thread-3) [n:127.0.0.1:63800_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:63800_solr
[junit4] 2> 2361237 INFO (jetty-launcher-2359-thread-1) [n:127.0.0.1:63803_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63803_solr
[junit4] 2> 2361239 INFO (jetty-launcher-2359-thread-3) [n:127.0.0.1:63800_solr ] o.a.s.c.Overseer Overseer (id=97439766102867974-127.0.0.1:63800_solr-n_0000000000) starting
[junit4] 2> 2361240 INFO (zkCallback-2376-thread-1-processing-n:127.0.0.1:63803_solr) [n:127.0.0.1:63803_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2361240 INFO (zkCallback-2372-thread-1-processing-n:127.0.0.1:63800_solr) [n:127.0.0.1:63800_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2361262 INFO (jetty-launcher-2359-thread-3) [n:127.0.0.1:63800_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63800_solr
[junit4] 2> 2361265 INFO (zkCallback-2376-thread-1-processing-n:127.0.0.1:63803_solr) [n:127.0.0.1:63803_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2361265 INFO (zkCallback-2372-thread-1-processing-n:127.0.0.1:63800_solr) [n:127.0.0.1:63800_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2361270 INFO (jetty-launcher-2359-thread-2) [n:127.0.0.1:63798_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2361300 INFO (jetty-launcher-2359-thread-2) [n:127.0.0.1:63798_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63798_solr
[junit4] 2> 2361303 INFO (zkCallback-2372-thread-2-processing-n:127.0.0.1:63800_solr) [n:127.0.0.1:63800_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2361303 INFO (zkCallback-2376-thread-1-processing-n:127.0.0.1:63803_solr) [n:127.0.0.1:63803_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2361304 INFO (zkCallback-2380-thread-1-processing-n:127.0.0.1:63798_solr) [n:127.0.0.1:63798_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2361308 INFO (jetty-launcher-2359-thread-4) [n:127.0.0.1:63799_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2361319 INFO (jetty-launcher-2359-thread-4) [n:127.0.0.1:63799_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63799_solr
[junit4] 2> 2361323 INFO (zkCallback-2380-thread-1-processing-n:127.0.0.1:63798_solr) [n:127.0.0.1:63798_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2361324 INFO (zkCallback-2379-thread-1-processing-n:127.0.0.1:63799_solr) [n:127.0.0.1:63799_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2361329 INFO (zkCallback-2376-thread-1-processing-n:127.0.0.1:63803_solr) [n:127.0.0.1:63803_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2361329 INFO (zkCallback-2372-thread-1-processing-n:127.0.0.1:63800_solr) [n:127.0.0.1:63800_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2362016 INFO (jetty-launcher-2359-thread-4) [n:127.0.0.1:63799_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node4/.
[junit4] 2> 2362016 INFO (jetty-launcher-2359-thread-2) [n:127.0.0.1:63798_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node2/.
[junit4] 2> 2362017 INFO (jetty-launcher-2359-thread-3) [n:127.0.0.1:63800_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node3/.
[junit4] 2> 2362018 INFO (jetty-launcher-2359-thread-1) [n:127.0.0.1:63803_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node1/.
[junit4] 2> 2364380 INFO (SUITE-CustomCollectionTest-seed#[167C8A2C2AD6C4E3]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2364383 INFO (SUITE-CustomCollectionTest-seed#[167C8A2C2AD6C4E3]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:63796/solr ready
[junit4] 2> 2364474 INFO (TEST-CustomCollectionTest.testCustomCollectionsAPI-seed#[167C8A2C2AD6C4E3]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCustomCollectionsAPI
[junit4] 2> 2364476 INFO (TEST-CustomCollectionTest.testCustomCollectionsAPI-seed#[167C8A2C2AD6C4E3]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2364482 INFO (qtp1731405362-14308) [n:127.0.0.1:63799_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params shards=a,b,c&replicationFactor=4&collection.configName=conf&maxShardsPerNode=5&name=implicitcoll&router.name=implicit&action=CREATE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2364486 INFO (OverseerThreadFactory-7617-thread-1-processing-n:127.0.0.1:63800_solr) [n:127.0.0.1:63800_solr ] o.a.s.c.CreateCollectionCmd Create collection implicitcoll
[junit4] 2> 2364608 INFO (qtp1731405362-14314) [n:127.0.0.1:63799_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=implicitcoll_b_replica3&action=CREATE&numShards=3&collection=implicitcoll&shard=b&wt=javabin&version=2
[junit4] 2> 2364611 INFO (qtp1731405362-14315) [n:127.0.0.1:63799_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=implicitcoll_a_replica3&action=CREATE&numShards=3&collection=implicitcoll&shard=a&wt=javabin&version=2
[junit4] 2> 2364611 INFO (qtp1731405362-14316) [n:127.0.0.1:63799_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=implicitcoll_c_replica3&action=CREATE&numShards=3&collection=implicitcoll&shard=c&wt=javabin&version=2
[junit4] 2> 2364614 INFO (qtp1433610066-14309) [n:127.0.0.1:63803_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=implicitcoll_a_replica4&action=CREATE&numShards=3&collection=implicitcoll&shard=a&wt=javabin&version=2
[junit4] 2> 2364614 INFO (qtp1433610066-14323) [n:127.0.0.1:63803_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=implicitcoll_b_replica4&action=CREATE&numShards=3&collection=implicitcoll&shard=b&wt=javabin&version=2
[junit4] 2> 2364615 INFO (qtp1433610066-14317) [n:127.0.0.1:63803_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=implicitcoll_c_replica4&action=CREATE&numShards=3&collection=implicitcoll&shard=c&wt=javabin&version=2
[junit4] 2> 2364619 INFO (qtp686265105-14298) [n:127.0.0.1:63798_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=implicitcoll_c_replica2&action=CREATE&numShards=3&collection=implicitcoll&shard=c&wt=javabin&version=2
[junit4] 2> 2364619 INFO (qtp686265105-14302) [n:127.0.0.1:63798_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=implicitcoll_b_replica2&action=CREATE&numShards=3&collection=implicitcoll&shard=b&wt=javabin&version=2
[junit4] 2> 2364620 INFO (qtp686265105-14303) [n:127.0.0.1:63798_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=implicitcoll_a_replica2&action=CREATE&numShards=3&collection=implicitcoll&shard=a&wt=javabin&version=2
[junit4] 2> 2364621 INFO (qtp767238905-14313) [n:127.0.0.1:63800_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=implicitcoll_c_replica1&action=CREATE&numShards=3&collection=implicitcoll&shard=c&wt=javabin&version=2
[junit4] 2> 2364621 INFO (qtp767238905-14320) [n:127.0.0.1:63800_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=implicitcoll_a_replica1&action=CREATE&numShards=3&collection=implicitcoll&shard=a&wt=javabin&version=2
[junit4] 2> 2364621 INFO (qtp767238905-14326) [n:127.0.0.1:63800_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=implicitcoll_b_replica1&action=CREATE&numShards=3&collection=implicitcoll&shard=b&wt=javabin&version=2
[junit4] 2> 2364772 INFO (zkCallback-2376-thread-1-processing-n:127.0.0.1:63803_solr) [n:127.0.0.1:63803_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/implicitcoll/state.json] for collection [implicitcoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2364772 INFO (zkCallback-2372-thread-1-processing-n:127.0.0.1:63800_solr) [n:127.0.0.1:63800_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/implicitcoll/state.json] for collection [implicitcoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2364772 INFO (zkCallback-2380-thread-1-processing-n:127.0.0.1:63798_solr) [n:127.0.0.1:63798_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/implicitcoll/state.json] for collection [implicitcoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2364772 INFO (zkCallback-2379-thread-1-processing-n:127.0.0.1:63799_solr) [n:127.0.0.1:63799_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/implicitcoll/state.json] for collection [implicitcoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2365654 INFO (qtp1731405362-14316) [n:127.0.0.1:63799_solr c:implicitcoll s:c x:implicitcoll_c_replica3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2365655 INFO (qtp1731405362-14314) [n:127.0.0.1:63799_solr c:implicitcoll s:b x:implicitcoll_b_replica3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2365655 INFO (qtp1731405362-14315) [n:127.0.0.1:63799_solr c:implicitcoll s:a x:implicitcoll_a_replica3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2365662 INFO (qtp1433610066-14317) [n:127.0.0.1:63803_solr c:implicitcoll s:c x:implicitcoll_c_replica4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2365662 INFO (qtp1433610066-14323) [n:127.0.0.1:63803_solr c:implicitcoll s:b x:implicitcoll_b_replica4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2365761 INFO (qtp1433610066-14309) [n:127.0.0.1:63803_solr c:implicitcoll s:a x:implicitcoll_a_replica4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2365763 INFO (qtp767238905-14326) [n:127.0.0.1:63800_solr c:implicitcoll s:b x:implicitcoll_b_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2365767 INFO (qtp686265105-14298) [n:127.0.0.1:63798_solr c:implicitcoll s:c x:implicitcoll_c_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2365769 INFO (qtp767238905-14320) [n:127.0.0.1:63800_solr c:implicitcoll s:a x:implicitcoll_a_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2365777 INFO (qtp767238905-14326) [n:127.0.0.1:63800_solr c:implicitcoll s:b x:implicitcoll_b_replica1] o.a.s.s.IndexSchema [implicitcoll_b_replica1] Schema name=cloud-dynamic
[junit4] 2> 2365779 INFO (qtp1433610066-14309) [n:127.0.0.1:63803_solr c:implicitcoll s:a x:implicitcoll_a_replica4] o.a.s.s.IndexSchema [implicitcoll_a_replica4] Schema name=cloud-dynamic
[junit4] 2> 2365785 INFO (qtp686265105-14298) [n:127.0.0.1:63798_solr c:implicitcoll s:c x:implicitcoll_c_replica2] o.a.s.s.IndexSchema [implicitcoll_c_replica2] Schema name=cloud-dynamic
[junit4] 2> 2365786 INFO (qtp767238905-14320) [n:127.0.0.1:63800_solr c:implicitcoll s:a x:implicitcoll_a_replica1] o.a.s.s.IndexSchema [implicitcoll_a_replica1] Schema name=cloud-dynamic
[junit4] 2> 2365847 INFO (qtp1731405362-14315) [n:127.0.0.1:63799_solr c:implicitcoll s:a x:implicitcoll_a_replica3] o.a.s.s.IndexSchema [implicitcoll_a_replica3] Schema name=cloud-dynamic
[junit4] 2> 2365850 INFO (qtp1731405362-14314) [n:127.0.0.1:63799_solr c:implicitcoll s:b x:implicitcoll_b_replica3] o.a.s.s.IndexSchema [implicitcoll_b_replica3] Schema name=cloud-dynamic
[junit4] 2> 2365847 INFO (qtp1731405362-14316) [n:127.0.0.1:63799_solr c:implicitcoll s:c x:implicitcoll_c_replica3] o.a.s.s.IndexSchema [implicitcoll_c_replica3] Schema name=cloud-dynamic
[junit4] 2> 2365904 INFO (qtp686265105-14302) [n:127.0.0.1:63798_solr c:implicitcoll s:b x:implicitcoll_b_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2365904 INFO (qtp686265105-14303) [n:127.0.0.1:63798_solr c:implicitcoll s:a x:implicitcoll_a_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2365904 INFO (qtp767238905-14313) [n:127.0.0.1:63800_solr c:implicitcoll s:c x:implicitcoll_c_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2365912 INFO (qtp1433610066-14317) [n:127.0.0.1:63803_solr c:implicitcoll s:c x:implicitcoll_c_replica4] o.a.s.s.IndexSchema [implicitcoll_c_replica4] Schema name=cloud-dynamic
[junit4] 2> 2365982 WARN (qtp767238905-14320) [n:127.0.0.1:63800_solr c:implicitcoll s:a x:implicitcoll_a_replica1] o.a.s.s.IndexSchema [implicitcoll_a_replica1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2365983 INFO (qtp767238905-14313) [n:127.0.0.1:63800_solr c:implicitcoll s:c x:implicitcoll_c_replica1] o.a.s.s.IndexSchema [implicitcoll_c_replica1] Schema name=cloud-dynamic
[junit4] 2> 2365983 INFO (qtp767238905-14320) [n:127.0.0.1:63800_solr c:implicitcoll s:a x:implicitcoll_a_replica1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 2365983 INFO (qtp1433610066-14323) [n:127.0.0.1:63803_solr c:implicitcoll s:b x:implicitcoll_b_replica4] o.a.s.s.IndexSchema [implicitcoll_b_replica4] Schema name=cloud-dynamic
[junit4] 2> 2365984 INFO (qtp686265105-14303) [n:127.0.0.1:63798_solr c:implicitcoll s:a x:implicitcoll_a_replica2] o.a.s.s.IndexSchema [implicitcoll_a_replica2] Schema name=cloud-dynamic
[junit4] 2> 2365985 WARN (qtp767238905-14326) [n:127.0.0.1:63800_solr c:implicitcoll s:b x:implicitcoll_b_replica1] o.a.s.s.IndexSchema [implicitcoll_b_replica1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2365985 INFO (qtp767238905-14326) [n:127.0.0.1:63800_solr c:implicitcoll s:b x:implicitcoll_b_replica1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 2365988 INFO (qtp686265105-14302) [n:127.0.0.1:63798_solr c:implicitcoll s:b x:implicitcoll_b_replica2] o.a.s.s.IndexSchema [implicitcoll_b_replica2] Schema name=cloud-dynamic
[junit4] 2> 2365993 WARN (qtp1433610066-14317) [n:127.0.0.1:63803_solr c:implicitcoll s:c x:implicitcoll_c_replica4] o.a.s.s.IndexSchema [implicitcoll_c_replica4] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2365994 INFO (qtp1433610066-14317) [n:127.0.0.1:63803_solr c:implicitcoll s:c x:implicitcoll_c_replica4] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 2365994 INFO (qtp767238905-14326) [n:127.0.0.1:63800_solr c:implicitcoll s:b x:implicitcoll_b_replica1] o.a.s.c.CoreContainer Creating SolrCore 'implicitcoll_b_replica1' using configuration from collection implicitcoll
[junit4] 2> 2365994 INFO (qtp767238905-14326) [n:127.0.0.1:63800_solr c:implicitcoll s:b x:implicitcoll_b_replica1] o.a.s.c.SolrCore [[implicitcoll_b_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node3/implicitcoll_b_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node3/./implicitcoll_b_replica1/data/]
[junit4] 2> 2365995 INFO (qtp1433610066-14317) [n:127.0.0.1:63803_solr c:implicitcoll s:c x:implicitcoll_c_replica4] o.a.s.c.CoreContainer Creating SolrCore 'implicitcoll_c_replica4' using configuration from collection implicitcoll
[junit4] 2> 2366001 INFO (qtp1433610066-14317) [n:127.0.0.1:63803_solr c:implicitcoll s:c x:implicitcoll_c_replica4] o.a.s.c.SolrCore [[implicitcoll_c_replica4] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node1/implicitcoll_c_replica4], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node1/./implicitcoll_c_replica4/data/]
[junit4] 2> 2366008 WARN (qtp767238905-14313) [n:127.0.0.1:63800_solr c:implicitcoll s:c x:implicitcoll_c_replica1] o.a.s.s.IndexSchema [implicitcoll_c_replica1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2366010 INFO (qtp767238905-14313) [n:127.0.0.1:63800_solr c:implicitcoll s:c x:implicitcoll_c_replica1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 2366012 INFO (qtp767238905-14313) [n:127.0.0.1:63800_solr c:implicitcoll s:c x:implicitcoll_c_replica1] o.a.s.c.CoreContainer Creating SolrCore 'implicitcoll_c_replica1' using configuration from collection implicitcoll
[junit4] 2> 2366012 INFO (qtp767238905-14313) [n:127.0.0.1:63800_solr c:implicitcoll s:c x:implicitcoll_c_replica1] o.a.s.c.SolrCore [[implicitcoll_c_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node3/implicitcoll_c_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node3/./implicitcoll_c_replica1/data/]
[junit4] 2> 2366024 INFO (qtp767238905-14320) [n:127.0.0.1:63800_solr c:implicitcoll s:a x:implicitcoll_a_replica1] o.a.s.c.CoreContainer Creating SolrCore 'implicitcoll_a_replica1' using configuration from collection implicitcoll
[junit4] 2> 2366024 INFO (qtp767238905-14320) [n:127.0.0.1:63800_solr c:implicitcoll s:a x:implicitcoll_a_replica1] o.a.s.c.SolrCore [[implicitcoll_a_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node3/implicitcoll_a_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node3/./implicitcoll_a_replica1/data/]
[junit4] 2> 2366075 WARN (qtp686265105-14303) [n:127.0.0.1:63798_solr c:implicitcoll s:a x:implicitcoll_a_replica2] o.a.s.s.IndexSchema [implicitcoll_a_replica2] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2366076 INFO (qtp686265105-14303) [n:127.0.0.1:63798_solr c:implicitcoll s:a x:implicitcoll_a_replica2] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 2366078 WARN (qtp1731405362-14315) [n:127.0.0.1:63799_solr c:implicitcoll s:a x:implicitcoll_a_replica3] o.a.s.s.IndexSchema [implicitcoll_a_replica3] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2366079 INFO (qtp1731405362-14315) [n:127.0.0.1:63799_solr c:implicitcoll s:a x:implicitcoll_a_replica3] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 2366079 WARN (qtp1731405362-14316) [n:127.0.0.1:63799_solr c:implicitcoll s:c x:implicitcoll_c_replica3] o.a.s.s.IndexSchema [implicitcoll_c_replica3] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2366080 WARN (qtp1433610066-14309) [n:127.0.0.1:63803_solr c:implicitcoll s:a x:implicitcoll_a_replica4] o.a.s.s.IndexSchema [implicitcoll_a_replica4] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2366080 INFO (qtp1731405362-14316) [n:127.0.0.1:63799_solr c:implicitcoll s:c x:implicitcoll_c_replica3] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 2366080 INFO (qtp686265105-14303) [n:127.0.0.1:63798_solr c:implicitcoll s:a x:implicitcoll_a_replica2] o.a.s.c.CoreContainer Creating SolrCore 'implicitcoll_a_replica2' using configuration from collection implicitcoll
[junit4] 2> 2366081 INFO (qtp1433610066-14309) [n:127.0.0.1:63803_solr c:implicitcoll s:a x:implicitcoll_a_replica4] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 2366081 WARN (qtp686265105-14298) [n:127.0.0.1:63798_solr c:implicitcoll s:c x:implicitcoll_c_replica2] o.a.s.s.IndexSchema [implicitcoll_c_replica2] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2366081 INFO (qtp1731405362-14315) [n:127.0.0.1:63799_solr c:implicitcoll s:a x:implicitcoll_a_replica3] o.a.s.c.CoreContainer Creating SolrCore 'implicitcoll_a_replica3' using configuration from collection implicitcoll
[junit4] 2> 2366082 INFO (qtp1433610066-14309) [n:127.0.0.1:63803_solr c:implicitcoll s:a x:implicitcoll_a_replica4] o.a.s.c.CoreContainer Creating SolrCore 'implicitcoll_a_replica4' using configuration from collection implicitcoll
[junit4] 2> 2366082 INFO (qtp1731405362-14315) [n:127.0.0.1:63799_solr c:implicitcoll s:a x:implicitcoll_a_replica3] o.a.s.c.SolrCore [[implicitcoll_a_replica3] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node4/implicitcoll_a_replica3], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node4/./implicitcoll_a_replica3/data/]
[junit4] 2> 2366082 INFO (qtp686265105-14298) [n:127.0.0.1:63798_solr c:implicitcoll s:c x:implicitcoll_c_replica2] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 2366082 INFO (qtp1433610066-14309) [n:127.0.0.1:63803_solr c:implicitcoll s:a x:implicitcoll_a_replica4] o.a.s.c.SolrCore [[implicitcoll_a_replica4] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node1/implicitcoll_a_replica4], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node1/./implicitcoll_a_replica4/data/]
[junit4] 2> 2366083 INFO (qtp686265105-14303) [n:127.0.0.1:63798_solr c:implicitcoll s:a x:implicitcoll_a_replica2] o.a.s.c.SolrCore [[implicitcoll_a_replica2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node2/implicitcoll_a_replica2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node2/./implicitcoll_a_replica2/data/]
[junit4] 2> 2366084 INFO (qtp686265105-14298) [n:127.0.0.1:63798_solr c:implicitcoll s:c x:implicitcoll_c_replica2] o.a.s.c.CoreContainer Creating SolrCore 'implicitcoll_c_replica2' using configuration from collection implicitcoll
[junit4] 2> 2366084 INFO (qtp1731405362-14316) [n:127.0.0.1:63799_solr c:implicitcoll s:c x:implicitcoll_c_replica3] o.a.s.c.CoreContainer Creating SolrCore 'implicitcoll_c_replica3' using configuration from collection implicitcoll
[junit4] 2> 2366084 INFO (qtp686265105-14298) [n:127.0.0.1:63798_solr c:implicitcoll s:c x:implicitcoll_c_replica2] o.a.s.c.SolrCore [[implicitcoll_c_replica2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node2/implicitcoll_c_replica2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node2/./implicitcoll_c_replica2/data/]
[junit4] 2> 2366084 INFO (qtp1731405362-14316) [n:127.0.0.1:63799_solr c:implicitcoll s:c x:implicitcoll_c_replica3] o.a.s.c.SolrCore [[implicitcoll_c_replica3] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node4/implicitcoll_c_replica3], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node4/./implicitcoll_c_replica3/data/]
[junit4] 2> 2366084 WARN (qtp1731405362-14314) [n:127.0.0.1:63799_solr c:implicitcoll s:b x:implicitcoll_b_replica3] o.a.s.s.IndexSchema [implicitcoll_b_replica3] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2366085 INFO (qtp1731405362-14314) [n:127.0.0.1:63799_solr c:implicitcoll s:b x:implicitcoll_b_replica3] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 2366091 WARN (qtp1433610066-14323) [n:127.0.0.1:63803_solr c:implicitcoll s:b x:implicitcoll_b_replica4] o.a.s.s.IndexSchema [implicitcoll_b_replica4] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2366092 INFO (qtp1433610066-14323) [n:127.0.0.1:63803_solr c:implicitcoll s:b x:implicitcoll_b_replica4] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 2366093 INFO (qtp1433610066-14323) [n:127.0.0.1:63803_solr c:implicitcoll s:b x:implicitcoll_b_replica4] o.a.s.c.CoreContainer Creating SolrCore 'implicitcoll_b_replica4' using configuration from collection implicitcoll
[junit4] 2> 2366094 INFO (qtp1433610066-14323) [n:127.0.0.1:63803_solr c:implicitcoll s:b x:implicitcoll_b_replica4] o.a.s.c.SolrCore [[implicitcoll_b_replica4] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node1/implicitcoll_b_replica4], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node1/./implicitcoll_b_replica4/data/]
[junit4] 2> 2366095 WARN (qtp686265105-14302) [n:127.0.0.1:63798_solr c:implicitcoll s:b x:implicitcoll_b_replica2] o.a.s.s.IndexSchema [implicitcoll_b_replica2] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2366095 INFO (qtp1731405362-14314) [n:127.0.0.1:63799_solr c:implicitcoll s:b x:implicitcoll_b_replica3] o.a.s.c.CoreContainer Creating SolrCore 'implicitcoll_b_replica3' using configuration from collection implicitcoll
[junit4] 2> 2366095 INFO (qtp1731405362-14314) [n:127.0.0.1:63799_solr c:implicitcoll s:b x:implicitcoll_b_replica3] o.a.s.c.SolrCore [[implicitcoll_b_replica3] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node4/implicitcoll_b_replica3], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node4/./implicitcoll_b_replica3/data/]
[junit4] 2> 2366095 INFO (qtp686265105-14302) [n:127.0.0.1:63798_solr c:implicitcoll s:b x:implicitcoll_b_replica2] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 2366096 INFO (qtp686265105-14302) [n:127.0.0.1:63798_solr c:implicitcoll s:b x:implicitcoll_b_replica2] o.a.s.c.CoreContainer Creating SolrCore 'implicitcoll_b_replica2' using configuration from collection implicitcoll
[junit4] 2> 2366097 INFO (qtp686265105-14302) [n:127.0.0.1:63798_solr c:implicitcoll s:b x:implicitcoll_b_replica2] o.a.s.c.SolrCore [[implicitcoll_b_replica2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node2/implicitcoll_b_replica2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001/tempDir-001/node2/./implicitcoll_b_replica2/data/]
[junit4] 2> 2366322 INFO (qtp767238905-14326) [n:127.0.0.1:63800_solr c:implicitcoll s:b x:implicitcoll_b_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2366322 INFO (qtp767238905-14326) [n:127.0.0.1:63800_solr c:implicitcoll s:b x:implicitcoll_b_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2366324 INFO (qtp767238905-14326) [n:127.0.0.1:63800_solr c:implicitcoll s:b x:implicitcoll_b_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2366324 INFO (qtp767238905-14326) [n:127.0.0.1:63800_solr c:implicitcoll s:b x:implicitcoll_b_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2366325 INFO (qtp1433610066-14317) [n:127.0.0.1:63803_solr c:implicitcoll s:c x:implicitcoll_c_replica4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2366325 INFO (qtp1433610066-14317) [n:127.0.0.1:63803_solr c:implicitcoll s:c x:implicitcoll_c_replica4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2366362 INFO (qtp767238905-14326) [n:127.0.0.1:63800_solr c:implicitcoll s:b x:implicitcoll_b_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2e4a98cd[implicitcoll_b_replica1] main]
[junit4] 2> 2366373 INFO (qtp767238905-14326) [n:127.0.0.1:63800_solr c:implicitcoll s:b x:implicitcoll_b_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2366374 INFO (qtp767238905-14326) [n:127.0.0.1:63800_solr c:implicitcoll s:b x:implicitcoll_b_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2366374 INFO (qtp767238905-14326) [n:127.0.0.1:63800_solr c:implicitcoll s:b x:implicitcoll_b_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2366375 INFO (searcherExecutor-7626-thread-1-processing-n:127.0.0.1:63800_solr x:implicitcoll_b_replica1 s:b c:implicitcoll) [n:127.0.0.1:63800_solr c:implicitcoll s:b x:implicitcoll_b_replica1] o.a.s.c.SolrCore [implicitcoll_b_replica1] Registered new searcher Searcher@2e4a98cd[implicitcoll_b_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2366375 INFO (qtp767238905-14326) [n:127.0.0.1:63800_solr c:implicitcoll s:b x:implicitcoll_b_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559036263232700416
[junit4] 2> 2366390 INFO (qtp1433610066-14323) [n:127.0.0.1:63803_solr c:implicitcoll s:b x:implicitcoll_b_replica4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2366390 INFO (qtp1433610066-14323) [n:127.0.0.1:63803_solr c:implicitcoll s:b x:implicitcoll_b_replica4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2366392 INFO (qtp1433610066-14323) [n:127.0.0.1:63803_solr c:implicitcoll s:b x:implicitcoll_b_replica4] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2366392 INFO (qtp1433610066-14323) [n:127.0.0.1:63803_solr c:implicitcoll s:b x:implicitcoll_b_replica4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2366404 INFO (qtp767238905-14320) [n:127.0.0.1:63800_solr c:implicitcoll s:a x:implicitcoll_a_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2366404 INFO (qtp767238905-14320) [n:127.0.0.1:63800_solr c:implicitcoll s:a x:implicitcoll_a_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2366406 INFO (qtp767238905-14320) [n:127.0.0.1:63800_solr c:implicitcoll s:a x:implicitcoll_a_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2366406 INFO (qtp767238905-14320) [n:127.0.0.1:63800_solr c:implicitcoll s:a x:implicitcoll_a_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2366408 INFO (qtp686265105-14298) [n:127.0.0.1:63798_solr c:implicitcoll s:c x:implicitcoll_c_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2366408 INFO (qtp686265105-14298) [n:127.0.0.1:63798_solr c:implicitcoll s:c x:implicitcoll_c_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2366410 INFO (qtp686265105-14298) [n:127.0.0.1:63798_solr c:implicitcoll s:c x:implicitcoll_c_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2366410 INFO (qtp686265105-14298) [n:127.0.0.1:63798_solr c:implicitcoll s:c x:implicitcoll_c_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2366413 INFO (qtp767238905-14326) [n:127.0.0.1:63800_solr c:implicitcoll s:b x:implicitcoll_b_replica1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard b: total=4 found=1 timeoutin=9999ms
[junit4] 2> 2366415 INFO (zkCallback-2376-thread-1-processing-n:127.0.0.1:63803_solr) [n:127.0.0.1:63803_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/implicitcoll/state.json] for collection [implicitcoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2366415 INFO (zkCallback-2379-thread-1-processing-n:127.0.0.1:63799_solr) [n:127.0.0.1:63799_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/implicitcoll/state.json] for collection [implicitcoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2366415 INFO (zkCallback-2372-thread-2-processing-n:127.0.0.1:63800_solr) [n:127.0.0.1:63800_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/implicitcoll/state.json] for collection [implicitcoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2366416 INFO (zkCallback-2380-thread-1-processing-n:127.0.0.1:63798_solr) [n:127.0.0.1:63798_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/implicitcoll/state.json] for collection [implicitcoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2366417 INFO (qtp1433610066-14323) [n:127.0.0.1:63803_solr c:implicitcoll s:b x:implicitcoll_b_replica4] o.a.s.s.SolrIndexSearcher Opening [Searcher@e2d3aa9[implicitcoll_b_replica4] main]
[junit4] 2> 2366423 INFO (qtp1433610066-14323) [n:127.0.0.1:63803_solr c:implicitcoll s:b x:implicitcoll_b_replica4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2366424 INFO (qtp1433610066-14323) [n:127.0.0.1:63803_solr c:implicitcoll s:b x:implicitcoll_b_replica4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2366424 INFO (qtp1433610066-14323) [n:127.0.0.1:63803_solr c:implicitcoll s:b x:implicitcoll_b_replica4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2366424 INFO (searcherExecutor-7635-thread-1-processing-n:127.0.0.1:63803_solr x:implicitcoll_b_replica4 s:b c:implicitcoll) [n:127.0.0.1:63803_solr c:implicitcoll s:b x:implicitcoll_b_replica4] o.a.s.c.SolrCore [implicitcoll_b_replica4] Registered new searcher Searcher@e2d3aa9[implicitcoll_b_replica4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2366424 INFO (qtp1433610066-14323) [n:127.0.0.1:63803_solr c:implicitcoll s:b x:implicitcoll_b_replica4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559036263284080640
[junit4] 2> 2366430 INFO (qtp767238905-14320) [n:127.0.0.1:63800_solr c:implicitcoll s:a x:implicitcoll_a_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@18334a14[implicitcoll_a_replica1] main]
[junit4] 2> 2366433 INFO (qtp767238905-14313) [n:127.0.0.1:63800_solr c:implicitcoll s:c x:implicitcoll_c_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2366433 INFO (qtp767238905-14313) [n:127.0.0.1:63800_solr c:implicitcoll s:c x:implicitcoll_c_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2366435 INFO (qtp767238905-14313) [n:127.0.0.1:63800_solr c:implicitcoll s:c x:implicitcoll_c_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2366435 INFO (qtp767238905-14313) [n:127.0.0.1:63800_solr c:implicitcoll s:c x:implicitcoll_c_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2366440 INFO (qtp1433610066-14317) [n:127.0.0.1:63803_solr c:implicitcoll s:c x:implicitcoll_c_replica4] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2366440 INFO (qtp1433610066-14317) [n:127.0.0.1:63803_solr c:implicitcoll s:c x:implicitcoll_c_replica4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2366451 INFO (qtp1731405362-14316) [n:127.0.0.1:63799_solr c:implicitcoll s:c x:implicitcoll_c_replica3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2366451 INFO (qtp1731405362-14316) [n:127.0.0.1:63799_solr c:implicitcoll s:c x:implicitcoll_c_replica3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2366451 INFO (qtp1731405362-14315) [n:127.0.0.1:63799_solr c:implicitcoll s:a x:implicitcoll_a_replica3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2366451 INFO (qtp1731405362-14315) [n:127.0.0.1:63799_solr c:implicitcoll s:a x:implicitcoll_a_replica3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2366451 INFO (qtp1433610066-14309) [n:127.0.0.1:63803_solr c:implicitcoll s:a x:implicitcoll_a_replica4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2366451 INFO (qtp1433610066-14309) [n:127.0.0.1:63803_solr c:implicitcoll s:a x:implicitcoll_a_replica4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2366456 INFO (qtp1433610066-14309) [n:127.0.0.1:63803_solr c:implicitcoll s:a x:implicitcoll_a_replica4] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2366456 INFO (qtp1433610066-14309) [n:127.0.0.1:63803_solr c:implicitcoll s:a x:implicitcoll_a_replica4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2366457 INFO (qtp1731405362-14316) [n:127.0.0.1:63799_solr c:implicitcoll s:c x:implicitcoll_c_replica3] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2366457 INFO (qtp1731405362-14316) [n:127.0.0.1:63799_solr c:implicitcoll s:c x:implicitcoll_c_replica3] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2366459 INFO (qtp686265105-14298) [n:127.0.0.1:63798_solr c:implicitcoll s:c x:implicitcoll_c_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@e9c2729[implicitcoll_c_replica2] main]
[junit4] 2> 2366460 INFO (qtp1731405362-14315) [n:127.0.0.1:63799_solr c:implicitcoll s:a x:implicitcoll_a_replica3] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2366460 INFO (qtp1731405362-14315) [n:127.0.0.1:63799_solr c:implicitcoll s:a x:implicitcoll_a_replica3] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2366462 INFO (qtp1433610066-14317) [n:127.0.0.1:63803_solr c:implicitcoll s:c x:implicitcoll_c_replica4] o.a.s.s.SolrIndexSearcher Opening [Searcher@6d7baf29[implicitcoll_c_replica4] main]
[junit4] 2> 2366464 INFO (qtp1433610066-14317) [n:127.0.0.1:63803_solr c:implicitcoll s:c x:implicitcoll_c_replica4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2366465 INFO (qtp1433610066-14317) [n:127.0.0.1:63803_solr c:implicitcoll s:c x:implicitcoll_c_replica4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2366465 INFO (qtp1433610066-14317) [n:127.0.0.1:63803_solr c:implicitcoll s:c x:implicitcoll_c_replica4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2366465 INFO (searcherExecutor-7627-thread-1-processing-n:127.0.0.1:63803_solr x:implicitcoll_c_replica4 s:c c:implicitcoll) [n:127.0.0.1:63803_solr c:implicitcoll s:c x:implicitcoll_c_replica4] o.a.s.c.SolrCore [implicitcoll_c_replica4] Registered new searcher Searcher@6d7baf29[implicitcoll_c_replica4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2366465 INFO (qtp1433610066-14317) [n:127.0.0.1:63803_solr c:implicitcoll s:c x:implicitcoll_c_replica4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559036263327072256
[junit4] 2> 2366466 INFO (qtp686265105-14302) [n:127.0.0.1:63798_solr c:implicitcoll s:b x:implicitcoll_b_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2366466 INFO (qtp686265105-14302) [n:127.0.0.1:63798_solr c:implicitcoll s:b x:implicitcoll_b_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2366470 INFO (qtp686265105-14302) [n:127.0.0.1:63798_solr c:implicitcoll s:b x:implicitcoll_b_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2366471 INFO (qtp686265105-14302) [n:127.0.0.1:63798_solr c:implicitcoll s:b x:implicitcoll_b_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2366471 INFO (qtp1731405362-14314) [n:127.0.0.1:63799_solr c:implicitcoll s:b x:implicitcoll_b_replica3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2366471 INFO (qtp1731405362-14314) [n:127.0.0.1:63799_solr c:implicitcoll s:b x:implicitcoll_b_replica3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2366473 INFO (qtp686265105-14298) [n:127.0.0.1:63798_solr c:implicitcoll s:c x:implicitcoll_c_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2366473 INFO (qtp1731405362-14314) [n:127.0.0.1:63799_solr c:implicitcoll s:b x:implicitcoll_b_replica3] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2366473 INFO (qtp1731405362-14314) [n:127.0.0.1:63799_solr c:implicitcoll s:b x:implicitcoll_b_replica3] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2366474 INFO (qtp767238905-14320) [n:127.0.0.1:63800_solr c:implicitcoll s:a x:implicitcoll_a_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2366474 INFO (qtp686265105-14298) [n:127.0.0.1:63798_solr c:implicitcoll s:c x:implicitcoll_c_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2366475 INFO (qtp686265105-14298) [n:127.0.0.1:63798_solr c:implicitcoll s:c x:implicitcoll_c_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2366475 INFO (searcherExecutor-7633-thread-1-processing-n:127.0.0.1:63798_solr x:implicitcoll_c_replica2 s:c c:implicitcoll) [n:127.0.0.1:63798_solr c:implicitcoll s:c x:implicitcoll_c_replica2] o.a.s.c.SolrCore [implicitcoll_c_replica2] Registered new searcher Searcher@e9c2729[implicitcoll_c_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2366475 INFO (qtp686265105-14298) [n:127.0.0.1:63798_solr c:implicitcoll s:c x:implicitcoll_c_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559036263337558016
[junit4] 2> 2366476 INFO (qtp1433610066-14317) [n:127.0.0.1:63803_solr c:implicitcoll s:c x:implicitcoll_c_replica4] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard c: total=4 found=1 timeoutin=9999ms
[junit4] 2> 2366476 INFO (qtp767238905-14313) [n:127.0.0.1:63800_solr c:implicitcoll s:c x:implicitcoll_c_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@311135db[implicitcoll_c_replica1] main]
[junit4] 2> 2366478 INFO (qtp1731405362-14316) [n:127.0.0.1:63799_solr c:implicitcoll s:c x:implicitcoll_c_replica3] o.a.s.s.SolrIndexSearcher Opening [Searcher@421522b2[implicitcoll_c_replica3] main]
[junit4] 2> 2366479 INFO (qtp1731405362-14316) [n:127.0.0.1:63799_solr c:implicitcoll s:c x:implicitcoll_c_replica3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2366480 INFO (qtp1731405362-14316) [n:127.0.0.1:63799_solr c:implicitcoll s:c x:implicitcoll_c_replica3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2366480 INFO (qtp1731405362-14316) [n:127.0.0.1:63799_solr c:implicitcoll s:c x:implicitcoll_c_replica3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2366480 INFO (qtp1731405362-14316) [n:127.0.0.1:63799_solr c:implicitcoll s:c x:implicitcoll_c_replica3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559036263342800896
[junit4] 2> 2366480 INFO (searcherExecutor-7634-thread-1-processing-n:127.0.0.1:63799_solr x:implicitcoll_c_replica3 s:c c:implicitcoll) [n:127.0.0.1:63799_solr c:implicitcoll s:c x:implicitcoll_c_replica3] o.a.s.c.SolrCore [implicitcoll_c_replica3] Registered new searcher Searcher@421522b2[implicitcoll_c_replica3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2366481 INFO (qtp1433610066-14309) [n:127.0.0.1:63803_solr c:implicitcoll s:a x:implicitcoll_a_replica4] o.a.s.s.SolrIndexSearcher Opening [Searcher@30551271[implicitcoll_a_replica4] main]
[junit4] 2> 2366482 INFO (qtp686265105-14303) [n:127.0.0.1:63798_solr c:implicitcoll s:a x:implicitcoll_a_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2366482 INFO (qtp686265105-14303) [n:127.0.0.1:63798_solr c:implicitcoll s:a x:implicitcoll_a_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2366484 INFO (qtp1433610066-14309) [n:127.0.0.1:63803_solr c:implicitcoll s:a x:implicitcoll_a_replica4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2366485 INFO (qtp1433610066-14309) [n:127.0.0.1:63803_solr c:implicitcoll s:a x:implicitcoll_a_replica4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2366485 INFO (qtp1433610066-14309) [n:127.0.0.1:63803_solr c:implicitcoll s:a x:implicitcoll_a_replica4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2366485 INFO (qtp1433610066-14309) [n:127.0.0.1:63803_solr c:implicitcoll s:a x:implicitcoll_a_replica4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559036263348043776
[junit4] 2> 2366485 INFO (searcherExecutor-7632-thread-1-processing-n:127.0.0.1:63803_solr x:implicitcoll_a_replica4 s:a c:implicitcoll) [n:127.0.0.1:63803_solr c:implicitcoll s:a x:implicitcoll_a_replica4] o.a.s.c.SolrCore [implicitcoll_a_replica4] Registered new searcher Searcher@30551271[implicitcoll_a_replica4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2366486 INFO (qtp686265105-14303) [n:127.0.0.1:63798_solr c:implicitcoll s:a x:implicitcoll_a_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2366486 INFO (qtp686265105-14303) [n:127.0.0.1:63798_solr c:implicitcoll s:a x:implicitcoll_a_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2366487 INFO (qtp767238905-14313) [n:127.0.0.1:63800_solr c:implicitcoll s:c x:implicitcoll_c_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2366487 INFO (qtp767238905-14320) [n:127.0.0.1:63800_solr c:implicitcoll s:a x:implicitcoll_a_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2366487 INFO (qtp767238905-14320) [n:127.0.0.1:63800_solr c:implicitcoll s:a x:implicitcoll_a_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2366488 INFO (qtp767238905-14313) [n:127.0.0.1:63800_solr c:implicitcoll s:c x:implicitcoll_c_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2366488 INFO (qtp767238905-14313) [n:127.0.0.1:63800_solr c:implicitcoll s:c x:implicitcoll_c_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2366488 INFO (qtp767238905-14313) [n:127.0.0.1:63800_solr c:implicitcoll s:c x:implicitcoll_c_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559036263351189504
[junit4] 2> 2366488 INFO (searcherExecutor-7628-thread-1-processing-n:127.0.0.1:63800_solr x:implicitcoll_c_replica1 s:c c:implicitcoll) [n:127.0.0.1:63800_solr c:implicitcoll s:c x:implicitcoll_c_replica1] o.a.s.c.SolrCore [implicitcoll_c_replica1] Registered new searcher Searcher@311135db[implicitcoll_c_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2366490 INFO (qtp1731405362-14315) [n:127.0.0.1:63799_solr c:implicitcoll s:a x:implicitcoll_a_replica3] o.a.s.s.SolrIndexSearcher Opening [Searcher@7730a229[implicitcoll_a_replica3] main]
[junit4] 2> 2366492 INFO (qtp1731405362-14315) [n:127.0.0.1:63799_solr c:implicitcoll s:a x:implicitcoll_a_replica3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2366492 INFO (qtp686265105-14302) [n:127.0.0.1:63798_solr c:implicitcoll s:b x:implicitcoll_b_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@4a039fb1[implicitcoll_b_replica2] main]
[junit4] 2> 2366493 INFO (qtp1731405362-14315) [n:127.0.0.1:63799_solr c:implicitcoll s:a x:implicitcoll_a_replica3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2366493 INFO (qtp1731405362-14315) [n:127.0.0.1:63799_solr c:implicitcoll s:a x:implicitcoll_a_replica3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2366493 INFO (searcherExecutor-7631-thread-1-processing-n:127.0.0.1:63799_solr x:implicitcoll_a_replica3 s:a c:implicitcoll) [n:127.0.0.1:63799_solr c:implicitcoll s:a x:implicitcoll_a_replica3] o.a.s.c.SolrCore [implicitcoll_a_replica3] Registered new searcher Searcher@7730a229[implicitcoll_a_replica3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2366493 INFO (qtp1731405362-14315) [n:127.0.0.1:63799_solr c:implicitcoll s:a x:implicitcoll_a_replica3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559036263356432384
[junit4] 2> 2366494 INFO (qtp686265105-14302) [n:127.0.0.1:63798_solr c:implicitcoll s:b x:implicitcoll_b_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2366494 INFO (qtp1731405362-14314) [n:127.0.0.1:63799_solr c:implicitcoll s:b x:implicitcoll_b_replica3] o.a.s.s.SolrIndexSearcher Opening [Searcher@7fe359a4[implicitcoll_b_replica3] main]
[junit4] 2> 2366495 INFO (qtp686265105-14302) [n:127.0.0.1:63798_solr c:implicitcoll s:b x:implicitcoll_b_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2366495 INFO (qtp686265105-14302) [n:127.0.0.1:63798_solr c:implicitcoll s:b x:implicitcoll_b_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2366495 INFO (qtp686265105-14302) [n:127.0.0.1:63798_solr c:implicitcoll s:b x:implicitcoll_b_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559036263358529536
[junit4] 2> 2366496 INFO (qtp1731405362-14314) [n:127.0.0.1:63799_solr c:implicitcoll s:b x:implicitcoll_b_replica3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2366495 INFO (searcherExecutor-7637-thread-1-processing-n:127.0.0.1:63798_solr x:implicitcoll_b_replica2 s:b c:implicitcoll) [n:127.0.0.1:63798_solr c:implicitcoll s:b x:implicitcoll_b_replica2] o.a.s.c.SolrCore [implicitcoll_b_replica2] Registered new searcher Searcher@4a039fb1[implicitcoll_b_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2366496 INFO (qtp1731405362-14314) [n:127.0.0.1:63799_solr c:implicitcoll s:b x:implicitcoll_b_replica3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2366497 INFO (qtp1731405362-14314) [n:127.0.0.1:63799_solr c:implicitcoll s:b x:implicitcoll_b_replica3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2366497 INFO (searcherExecutor-7636-thread-1-processing-n:127.0.0.1:63799_solr x:implicitcoll_b_replica3 s:b c:implicitcoll) [n:127.0.0.1:63799_solr c:implicitcoll s:b x:implicitcoll_b_replica3] o.a.s.c.SolrCore [implicitcoll_b_replica3] Registered new searcher Searcher@7fe359a4[implicitcoll_b_replica3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2366497 INFO (qtp1731405362-14314) [n:127.0.0.1:63799_solr c:implicitcoll s:b x:implicitcoll_b_replica3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559036263360626688
[junit4] 2> 2366501 INFO (qtp1433610066-14309) [n:127.0.0.1:63803_solr c:implicitcoll s:a x:implicitcoll_a_replica4] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard a: total=4 found=1 timeoutin=9998ms
[junit4] 2> 2366503 INFO (searcherExecutor-7629-thread-1-processing-n:127.0.0.1:63800_solr x:implicitcoll_a_replica1 s:a c:implicitcoll) [n:127.0.0.1:63800_solr c:implicitcoll s:a x:implicitcoll_a_replica1] o.a.s.c.SolrCore [implicitcoll_a_replica1] Registered new searcher Searcher@18334a14[implicitcoll_a_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2366503 INFO (qtp767238905-14320) [n:127.0.0.1:63800_solr c:implicitcoll s:a x:implicitcoll_a_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559036263366918144
[junit4] 2> 2366507 INFO (qtp686265105-14303) [n:127.0.0.1:63798_solr c:implicitcoll s:a x:implicitcoll_a_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@353dc10e[implicitcoll_a_replica2] main]
[junit4] 2> 2366508 INFO (qtp686265105-14303) [n:127.0.0.1:63798_solr c:implicitcoll s:a x:implicitcoll_a_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2366509 INFO (qtp686265105-14303) [n:127.0.0.1:63798_solr c:implicitcoll s:a x:implicitcoll_a_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2366509 INFO (qtp686265105-14303) [n:127.0.0.1:63798_solr c:implicitcoll s:a x:implicitcoll_a_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2366509 INFO (qtp686265105-14303) [n:127.0.0.1:63798_solr c:implicitcoll s:a x:implicitcoll_a_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559036263373209600
[junit4] 2> 2366509 INFO (searcherExecutor-7630-thread-1-processing-n:127.0.0.1:63798_solr x:implicitcoll_a_replica2 s:a c:implicitcoll) [n:127.0.0.1:63798_solr c:implicitcoll s:a x:implicitcoll_a_replica2] o.a.s.c.SolrCore [implicitcoll_a_replica2] Registered new searcher Searcher@353dc10e[implicitcoll_a_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2366600 INFO (zkCallback-2379-thread-1-processing-n:127.0.0.1:63799_solr) [n:127.0.0.1:63799_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/implicitcoll/state.json] for collection [implicitcoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2366600 INFO (zkCallback-2380-thread-1-processing-n:127.0.0.1:63798_solr) [n:127.0.0.1:63798_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/implicitcoll/state.json] for collection [implicitcoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2366600 INFO (zkCallback-2372-thread-2-processing-n:127.0.0.1:63800_solr) [n:127.0.0.1:63800_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/implicitcoll/state.json] for collection [implicitcoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2366600 INFO (zkCallback-2376-thread-1-processing-n:127.0.0.1:63803_solr) [n:127.0.0.1:63803_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/implicitcoll/state.json] for collection [implicitcoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2366922 INFO (qtp767238905-14326) [n:127.0.0.1:63800_solr c:implicitcoll s:b x:implicitcoll_b_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2366922 INFO (qtp767238905-14326) [n:127.0.0.1:63800_solr c:implicitcoll s:b x:implicitcoll_b_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2366922 INFO (qtp767238905-14326) [n:127.0.0.1:63800_solr c:implicitcoll s:b x:implicitcoll_b_replica1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:63800/solr/implicitcoll_b_replica1/
[junit4] 2> 2366923 INFO (qtp767238905-14326) [n:127.0.0.1:63800_solr c:implicitcoll s:b x:implicitcoll_b_replica1] o.a.s.u.PeerSync PeerSync: core=implicitcoll_b_replica1 url=http://127.0.0.1:63800/solr START replicas=[http://127.0.0.1:63799/solr/implicitcoll_b_replica3/, http://127.0.0.1:63803/solr/implicitcoll_b_replica4/, http://127.0.0.1:63798/solr/implicitcoll_b_replica2/] nUpdates=100
[junit4] 2> 2366924 INFO (qtp767238905-14326) [n:127.0.0.1:63800_solr c:implicitcoll s:b x:implicitcoll_b_replica1] o.a.s.u.PeerSync PeerSync: core=implicitcoll_b_replica1 url=http://127.0.0.1:63800/solr DONE. We have no versions. sync failed.
[junit4] 2> 2366926 INFO (qtp1731405362-14319) [n:127.0.0.1:63799_solr c:implicitcoll s:b r:core_node1 x:implicitcoll_b_replica3] o.a.s.c.S.Request [implicitcoll_b_replica3] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2366926 INFO (qtp686265105-14304) [n:127.0.0.1:63798_solr c:implicitcoll s:b r:core_node8 x:implicitcoll_b_replica2] o.a.s.c.S.Request [implicitcoll_b_replica2] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2366926 INFO (qtp1433610066-14324) [n:127.0.0.1:63803_solr c:implicitcoll s:b r:core_node5 x:implicitcoll_b_replica4] o.a.s.c.S.Request [implicitcoll_b_replica4] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2366927 INFO (qtp767238905-14326) [n:127.0.0.1:63800_solr c:implicitcoll s:b x:implicitcoll_b_replica1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 2366927 INFO (qtp767238905-14326) [n:127.0.0.1:63800_solr c:implicitcoll s:b x:implicitcoll_b_replica1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 2366935 INFO (qtp767238905-14326) [n:127.0.0.1:63800_solr c:implicitcoll s:b x:implicitcoll_b_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:63800/solr/implicitcoll_b_replica1/ b
[junit4] 2> 2366977 INFO (qtp1433610066-14317) [n:127.0.0.1:63803_solr c:implicitcoll s:c x:implicitcoll_c_replica4] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2366977 INFO (qtp1433610066-14317) [n:127.0.0.1:63803_solr c:implicitcoll s:c x:implicitcoll_c_replica4] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2366977 INFO (qtp1433610066-14317) [n:127.0.0.1:63803_solr c:implicitcoll s:c x:implicitcoll_c_replica4] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:63803/solr/implicitcoll_c_replica4/
[junit4] 2> 2366978 INFO (qtp1433610066-14317) [n:127.0.0.1:63803_solr c:implicitcoll s:c x:implicitcoll_c_replica4] o.a.s.u.PeerSync PeerSync: core=implicitcoll_c_replica4 url=http://127.0.0.1:63803/solr START replicas=[http://127.0.0.1:63799/solr/implicitcoll_c_replica3/, http://127.0.0.1:63798/solr/implicitcoll_c_replica2/, http://127.0.0.1:63800/solr/implicitcoll_c_replica1/] nUpdates=100
[junit4] 2> 2366979 INFO (qtp1433610066-14317) [n:127.0.0.1:63803_solr c:implicitcoll s:c x:implicitcoll_c_replica4] o.a.s.u.PeerSync PeerSync: core=implicitcoll_c_replica4 url=http://127.0.0.1:63803/solr DONE. We have no versions. sync failed.
[junit4] 2> 2366984 INFO (qtp686265105-14304) [n:127.0.0.1:63798_solr c:implicitcoll s:c r:core_node7 x:implicitcoll_c_replica2] o.a.s.c.S.Request [implicitcoll_c_replica2] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2366984 INFO (qtp767238905-14327) [n:127.0.0.1:63800_solr c:implicitcoll s:c r:core_node10 x:implicitcoll_c_replica1] o.a.s.c.S.Request [implicitcoll_c_replica1] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2366986 INFO (qtp1731405362-14321) [n:127.0.0.1:63799_solr c:implicitcoll s:c r:core_node4 x:implicitcoll_c_replica3] o.a.s.c.S.Request [implicitcoll_c_replica3] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2366986 INFO (qtp1433610066-14317) [n:127.0.0.1:63803_solr c:implicitcoll s:c x:implicitcoll_c_replica4] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 2366986 INFO (qtp1433610066-14317) [n:127.0.0.1:63803_solr c:implicitcoll s:c x:implicitcoll_c_replica4] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 2366998 INFO (qtp1433610066-14317) [n:127.0.0.1:63803_solr c:implicitcoll s:c x:implicitcoll_c_replica4] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:63803/solr/implicitcoll_c_replica4/ c
[junit4] 2> 2367002 INFO (qtp1433610066-14309) [n:127.0.0.1:63803_solr c:implicitcoll s:a x:implicitcoll_a_replica4] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2367003 INFO (qtp1433610066-14309) [n:127.0.0.1:63803_solr c:implicitcoll s:a x:implicitcoll_a_replica4] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2367003 INFO (qtp1433610066-14309) [n:127.0.0.1:63803_solr c:implicitcoll s:a x:implicitcoll_a_replica4] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:63803/solr/implicitcoll_a_replica4/
[junit4] 2> 2367003 INFO (qtp1433610066-14309) [n:127.0.0.1:63803_solr c:implicitcoll s:a x:implicitcoll_a_replica4] o.a.s.u.PeerSync PeerSync: core=implicitcoll_a_replica4 url=http://127.0.0.1:63803/solr START replicas=[http://127.0.0.1:63799/solr/implicitcoll_a_replica3/, http://127.0.0.1:63798/solr/implicitcoll_a_replica2/, http://127.0.0.1:63800/solr/implicitcoll_a_replica1/] nUpdates=100
[junit4] 2> 2367004 INFO (qtp1433610066-14309) [n:127.0.0.1:63803_solr c:implicitcoll s:a x:implicitcoll_a_replica4] o.a.s.u.PeerSync PeerSync: core=implicitcoll_a_replica4 url=http://127.0.0.1:63803/solr DONE. We have no versions. sync failed.
[junit4] 2> 2367004 INFO (qtp1731405362-14305) [n:127.0.0.1:63799_solr c:implicitcoll s:a r:core_node2 x:implicitcoll_a_replica3] o.a.s.c.S.Request [implicitcoll_a_replica3] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2367004 INFO (qtp686265105-14304) [n:127.0.0.1:63798_solr c:implicitcoll s:a r:core_node9 x:implicitcoll_a_replica2] o.a.s.c.S.Request [implicitcoll_a_replica2] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2367004 INFO (qtp767238905-14327) [n:127.0.0.1:63800_solr c:implicitcoll s:a r:core_node11 x:implicitcoll_a_replica1] o.a.s.c.S.Request [implicitcoll_a_replica1] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2367005 INFO (qtp1433610066-14309) [n:127.0.0.1:63803_solr c:implicitcoll s:a x:implicitcoll_a_replica4] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 2367005 INFO (qtp1433610066-14309) [n:127.0.0.1:63803_solr c:implicitcoll s:a x:implicitcoll_a_replica4] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 2367015 INFO (qtp1433610066-14309) [n:127.0.0.1:63803_solr c:implicitcoll s:a x:implicitcoll_a_replica4] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:63803/solr/implicitcoll_a_replica4/ a
[junit4] 2> 2367123 INFO (zkCallback-2376-thread-1-processing-n:127.0.0.1:63803_solr) [n:127.0.0.1:63803_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/implicitcoll/state.json] for collection [implicitcoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2367123 INFO (zkCallback-2372-thread-3-processing-n:127.0.0.1:63800_solr) [n:127.0.0.1:63800_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/implicitcoll/state.json] for collection [implicitcoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2367123 INFO (zkCallback-2380-thread-1-processing-n:127.0.0.1:63798_solr) [n:127.0.0.1:63798_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/implicitcoll/state.json] for collection [implicitcoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2367123 INFO (zkCallback-2379-thread-1-processing-n:127.0.0.1:63799_solr) [n:127.0.0.1:63799_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/implicitcoll/state.json] for collection [implicitcoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2367149 INFO (qtp767238905-14326) [n:127.0.0.1:63800_solr c:implicitcoll s:b x:implicitcoll_b_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2367154 INFO (qtp767238905-14326) [n:127.0.0.1:63800_solr c:implicitcoll s:b x:implicitcoll_b_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=implicitcoll_b_replica1&action=CREATE&numShards=3&collection=implicitcoll&shard=b&wt=javabin&version=2} status=0 QTime=2532
[junit4] 2> 2367166 INFO (qtp1433610066-14317) [n:127.0.0.1:63803_solr c:implicitcoll s:c x:implicitcoll_c_replica4] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2367169 INFO (qtp1433610066-14317) [n:127.0.0.1:63803_solr c:implicitcoll s:c x:implicitcoll_c_replica4] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=implicitcoll_c_replica4&action=CREATE&numShards=3&collection=implicitcoll&shard=c&wt=javabin&version=2} status=0 QTime=2554
[junit4] 2> 2367180 INFO (qtp1433610066-14309) [n:127.0.0.1:63803_solr c:implicitcoll s:a x:implicitcoll_a_replica4] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2367184 INFO (qtp1433610066-14309) [n:127.0.0.1:63803_solr c:implicitcoll s:a x:implicitcoll_a_replica4] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=implicitcoll_a_replica4&action=CREATE&numShards=3&collection=implicitcoll&shard=a&wt=javabin&version=2} status=0 QTime=2569
[junit4] 2> 2367296 INFO (zkCallback-2376-thread-1-processing-n:127.0.0.1:63803_solr) [n:127.0.0.1:63803_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/implicitcoll/state.json] for collection [implicitcoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2367296 INFO (zkCallback-2379-thread-1-processing-n:127.0.0.1:63799_solr) [n:127.0.0.1:63799_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/implicitcoll/state.json] for collection [implicitcoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2367296 INFO (zkCallback-2380-thread-1-processing-n:127.0.0.1:63798_solr) [n:127.0.0.1:63798_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/implicitcoll/state.json] for collection [implicitcoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2367296 INFO (zkCallback-2372-thread-3-processing-n:127.0.0.1:63800_solr) [n:127.0.0.1:63800_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/implicitcoll/state.json] for collection [implicitcoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2367469 INFO (qtp1433610066-14323) [n:127.0.0.1:63803_solr c:implicitcoll s:b x:implicitcoll_b_replica4] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=implicitcoll_b_replica4&action=CREATE&numShards=3&collection=implicitcoll&shard=b&wt=javabin&version=2} status=0 QTime=2854
[junit4] 2> 2367491 INFO (qtp1731405362-14316) [n:127.0.0.1:63799_solr c:implicitcoll s:c x:implicitcoll_c_replica3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=implicitcoll_c_replica3&action=CREATE&numShards=3&collection=implicitcoll&shard=c&wt=javabin&version=2} status=0 QTime=2880
[junit4] 2> 2367495 INFO (qtp686265105-14298) [n:127.0.0.1:63798_solr c:implicitcoll s:c x:implicitcoll_c_replica2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=implicitcoll_c_replica2&action=CREATE&numShards=3&collection=implicitcoll&shard=c&wt=javabin&version=2} status=0 QTime=2875
[junit4] 2> 2367509 INFO (qtp1731405362-14314) [n:127.0.0.1:63799_solr c:implicitcoll s:b x:implicitcoll_b_replica3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=implicitcoll_b_replica3&action=CREATE&numShards=3&collection=implicitcoll&shard=b&wt=javabin&version=2} status=0 QTime=2901
[junit4] 2> 2367509 INFO (qtp686265105-14302) [n:127.0.0.1:63798_solr c:implicitcoll s:b x:implicitcoll_b_replica2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=implicitcoll_b_replica2&action=CREATE&numShards=3&collection=implicitcoll&shard=b&wt=javabin&version=2} status=0 QTime=2889
[junit4] 2> 2367512 INFO (qtp1731405362-14315) [n:127.0.0.1:63799_
[...truncated too long message...]
ic reporters for: solr.node
[junit4] 2> 2423789 INFO (jetty-launcher-2359-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@12a19f46{/solr,null,UNAVAILABLE}
[junit4] 2> 2423801 INFO (jetty-launcher-2359-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 2423802 INFO (jetty-launcher-2359-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@540c4a5d{/solr,null,UNAVAILABLE}
[junit4] 2> 2423804 INFO (SUITE-CustomCollectionTest-seed#[167C8A2C2AD6C4E3]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:63796 63796
[junit4] 2> 2423805 INFO (Thread-3217) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:63796 63796
[junit4] 2> 2427267 WARN (Thread-3217) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 13 /solr/configs/conf
[junit4] 2> 9 /solr/clusterprops.json
[junit4] 2> 9 /solr/aliases.json
[junit4] 2> 4 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 9 /solr/clusterstate.json
[junit4] 2> 5 /solr/collections/implicitcoll/state.json
[junit4] 2> 4 /solr/collections/testCreateShardRepFactor/state.json
[junit4] 2> 4 /solr/collections/routeFieldColl/state.json
[junit4] 2> 4 /solr/collections/withShardField/state.json
[junit4] 2> 2 /solr/collections/implicitcoll/leader_elect/c/election/97439766102867976-core_node6-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 78 /solr/overseer/queue
[junit4] 2> 45 /solr/overseer/collection-queue-work
[junit4] 2> 27 /solr/collections
[junit4] 2> 9 /solr/live_nodes
[junit4] 2> 7 /solr/overseer/queue-work
[junit4] 2>
[junit4] 2> 2427267 INFO (SUITE-CustomCollectionTest-seed#[167C8A2C2AD6C4E3]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_167C8A2C2AD6C4E3-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70), sim=RandomSimilarity(queryNorm=false): {}, locale=he-IL, timezone=America/Port-au-Prince
[junit4] 2> NOTE: Mac OS X 10.11.6 x86_64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=3,threads=1,free=178633480,total=510947328
[junit4] 2> NOTE: All tests run in this JVM: [TestDFRSimilarityFactory, TestJettySolrRunner, CachingDirectoryFactoryTest, TestImplicitCoreProperties, MultiThreadedOCPTest, SpatialRPTFieldTypeTest, NoCacheHeaderTest, DistributedQueueTest, CacheHeaderTest, DistributedSuggestComponentTest, MigrateRouteKeyTest, TestTrieFacet, TestLMDirichletSimilarityFactory, LeaderElectionIntegrationTest, CdcrRequestHandlerTest, CloudExitableDirectoryReaderTest, BufferStoreTest, TestWriterPerf, TestDistributedGrouping, TestMaxScoreQueryParser, DistributedDebugComponentTest, TestSSLRandomization, TestQuerySenderListener, TestSolrDeletionPolicy2, TestCloudPseudoReturnFields, TestSolrCoreSnapshots, BaseCdcrDistributedZkTest, TestJoin, TestRawResponseWriter, TestReversedWildcardFilterFactory, BadComponentTest, MetricsHandlerTest, TestNonDefinedSimilarityFactory, VMParamsZkACLAndCredentialsProvidersTest, BasicAuthIntegrationTest, PreAnalyzedFieldTest, RollingRestartTest, TestManagedResourceStorage, FullHLLTest, FieldMutatingUpdateProcessorTest, ConvertedLegacyTest, TestMiniSolrCloudClusterSSL, TestStressRecovery, BitVectorTest, DataDrivenBlockJoinTest, LoggingHandlerTest, SuggesterTest, CloudMLTQParserTest, TlogReplayBufferedWhileIndexingTest, ActionThrottleTest, FullSolrCloudDistribCmdsTest, PKIAuthenticationIntegrationTest, TestSubQueryTransformerDistrib, WordBreakSolrSpellCheckerTest, TestRandomFaceting, TestDistributedMissingSort, TestRangeQuery, DistributedQueryComponentCustomSortTest, TestConfigsApi, CSVRequestHandlerTest, BasicDistributedZk2Test, TestSmileRequest, TestUninvertingReader, PeerSyncReplicationTest, HdfsRecoveryZkTest, TestAuthenticationFramework, URLClassifyProcessorTest, DateMathParserTest, CollectionsAPIAsyncDistributedZkTest, CircularListTest, TestSolrXml, BadIndexSchemaTest, TestSQLHandler, TestSystemCollAutoCreate, OverseerModifyCollectionTest, TestCustomStream, TestInitQParser, TestSolrCloudWithSecureImpersonation, CoreAdminHandlerTest, TestSimpleTrackingShardHandler, TestFieldTypeCollectionResource, TestScoreJoinQPScore, TestGeoJSONResponseWriter, SpellCheckComponentTest, SolrIndexMetricsTest, CollectionStateFormat2Test, TestBlendedInfixSuggestions, ManagedSchemaRoundRobinCloudTest, TestSearchPerf, TestConfig, TestSystemIdResolver, UUIDUpdateProcessorFallbackTest, SolrGangliaReporterTest, TestClusterProperties, AddSchemaFieldsUpdateProcessorFactoryTest, DeleteStatusTest, TermVectorComponentTest, CdcrReplicationDistributedZkTest, MetricUtilsTest, TestRealTimeGet, TestFoldingMultitermQuery, SharedFSAutoReplicaFailoverTest, ConcurrentDeleteAndCreateCollectionTest, TestLocalFSCloudBackupRestore, TestSolrQueryResponse, TestClassicSimilarityFactory, RuleEngineTest, HdfsWriteToMultipleCollectionsTest, TestInitParams, DistributedTermsComponentTest, TestSha256AuthenticationProvider, TestReqParamsAPI, TestFastWriter, TestConfigSetImmutable, SolrJmxReporterTest, RequestHandlersTest, BasicFunctionalityTest, ClassificationUpdateProcessorFactoryTest, UpdateLogTest, TestAtomicUpdateErrorCases, SaslZkACLProviderTest, TestIndexingPerformance, TestSolrQueryParser, TestOmitPositions, TestBulkSchemaAPI, TestManagedStopFilterFactory, BlockJoinFacetRandomTest, TestSchemaManager, UpdateParamsTest, DistributedIntervalFacetingTest, OverriddenZkACLAndCredentialsProvidersTest, OpenCloseCoreStressTest, DistributedFacetPivotSmallTest, TestTestInjection, ConjunctionSolrSpellCheckerTest, GraphQueryTest, TestQueryUtils, TestSubQueryTransformer, InfixSuggestersTest, SolrInfoMBeanTest, TestSuggestSpellingConverter, TestIndexSearcher, SliceStateTest, SignatureUpdateProcessorFactoryTest, TestBackupRepositoryFactory, AnalysisErrorHandlingTest, CoreMergeIndexesAdminHandlerTest, RankQueryTest, TestJavabinTupleStreamParser, TestSolrIndexConfig, SpellCheckCollatorTest, TestConfigSets, DistanceFunctionTest, TestDynamicFieldCollectionResource, TestExceedMaxTermLength, TestUseDocValuesAsStored, TestGraphTermsQParserPlugin, DefaultValueUpdateProcessorTest, TestIntervalFaceting, ShowFileRequestHandlerTest, TestComplexPhraseQParserPlugin, TestCodecSupport, ShufflingReplicaListTransformerTest, BlobRepositoryCloudTest, TestClusterStateMutator, TestHdfsCloudBackupRestore, TestSchemaNameResource, TestQueryWrapperFilter, ChaosMonkeySafeLeaderTest, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest, SyncSliceTest, BasicZkTest, RecoveryZkTest, ClusterStateUpdateTest, ZkSolrClientTest, TestZkChroot, ShardRoutingCustomTest, TestDistributedSearch, TestHashPartitioner, TermVectorComponentDistributedTest, HardAutoCommitTest, TestSolr4Spatial, SolrCmdDistributorTest, TestLazyCores, SolrIndexSplitterTest, TestCoreDiscovery, SuggesterTSTTest, SpatialFilterTest, TestCSVLoader, SolrCoreCheckLockOnStartupTest, TestPseudoReturnFields, SortByFunctionTest, TestSolrDeletionPolicy1, DebugComponentTest, TestSurroundQueryParser, LukeRequestHandlerTest, TestValueSourceCache, MoreLikeThisHandlerTest, FieldAnalysisRequestHandlerTest, RegexBoostProcessorTest, TestCollationField, UpdateRequestProcessorFactoryTest, QueryParsingTest, UniqFieldsUpdateProcessorFactoryTest, JsonLoaderTest, TestPartialUpdateDeduplication, SearchHandlerTest, HighlighterConfigTest, AlternateDirectoryTest, CopyFieldTest, ResponseLogComponentTest, SolrIndexConfigTest, MinimalSchemaTest, OutputWriterTest, TestSolrCoreProperties, SpellPossibilityIteratorTest, TestCharFilters, EchoParamsTest, TestLMJelinekMercerSimilarityFactory, ResourceLoaderTest, TestFastOutputStream, PluginInfoTest, DOMUtilTest, ClusterStateTest, ZkNodePropsTest, DistributedMLTComponentTest, SolrTestCaseJ4Test, TestCrossCoreJoin, TestCursorMarkWithoutUniqueKey, TestTolerantSearch, TestPathTrie, AssignTest, CdcrBootstrapTest, CollectionReloadTest, CollectionTooManyReplicasTest, ConfigSetsAPITest, ConnectionManagerTest, CustomCollectionTest]
[junit4] Completed [489/694 (1!)] on J1 in 66.26s, 4 tests, 1 error <<< FAILURES!
[...truncated 62889 lines...]
-documentation-lint:
[jtidy] Checking for broken html (such as invalid tags)...
[delete] Deleting directory /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/build/jtidy_tmp
[echo] Checking for broken links...
[exec]
[exec] Crawl/parse...
[exec]
[exec] Verify...
[echo] Checking for malformed docs...
[exec]
[exec] /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/docs/solr-solrj/overview-summary.html
[exec] missing: org.apache.solr.client.solrj.io.eval
[exec]
[exec] Missing javadocs were found!
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/build.xml:775: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/build.xml:101: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build.xml:654: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build.xml:670: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/common-build.xml:2493: exec returned: 1
Total time: 114 minutes 30 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3824
- Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3824/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC
All tests passed
Build Log:
[...truncated 68119 lines...]
-ecj-javadoc-lint-tests:
[mkdir] Created dir: /var/folders/qg/h2dfw5s161s51l2bn79mrb7r0000gn/T/ecj427903614
[ecj-lint] Compiling 763 source files to /var/folders/qg/h2dfw5s161s51l2bn79mrb7r0000gn/T/ecj427903614
[ecj-lint] invalid Class-Path header in manifest of jar file: /Users/jenkins/.ivy2/cache/org.restlet.jee/org.restlet/jars/org.restlet-2.3.0.jar
[ecj-lint] invalid Class-Path header in manifest of jar file: /Users/jenkins/.ivy2/cache/org.restlet.jee/org.restlet.ext.servlet/jars/org.restlet.ext.servlet-2.3.0.jar
[ecj-lint] ----------
[ecj-lint] 1. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/cloud/OverseerTest.java (at line 266)
[ecj-lint] ZkStateReader reader = new ZkStateReader(zkClient);
[ecj-lint] ^^^^^^
[ecj-lint] Resource leak: 'reader' is never closed
[ecj-lint] ----------
[ecj-lint] 2. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/cloud/OverseerTest.java (at line 316)
[ecj-lint] ZkStateReader reader = new ZkStateReader(zkClient);
[ecj-lint] ^^^^^^
[ecj-lint] Resource leak: 'reader' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 3. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/core/HdfsDirectoryFactoryTest.java (at line 146)
[ecj-lint] HdfsDirectoryFactory hdfsFactory = new HdfsDirectoryFactory();
[ecj-lint] ^^^^^^^^^^^
[ecj-lint] Resource leak: 'hdfsFactory' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 4. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/handler/admin/SecurityConfHandlerTest.java (at line 53)
[ecj-lint] BasicAuthPlugin basicAuth = new BasicAuthPlugin();
[ecj-lint] ^^^^^^^^^
[ecj-lint] Resource leak: 'basicAuth' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 5. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/handler/component/DistributedDebugComponentTest.java (at line 163)
[ecj-lint] SolrClient client = random().nextBoolean() ? collection1 : collection2;
[ecj-lint] ^^^^^^
[ecj-lint] Resource leak: 'client' is never closed
[ecj-lint] ----------
[ecj-lint] 6. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/handler/component/DistributedDebugComponentTest.java (at line 221)
[ecj-lint] throw new AssertionError(q.toString() + ": " + e.getMessage(), e);
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'client' is not closed at this location
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 7. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/highlight/HighlighterTest.java (at line 201)
[ecj-lint] Analyzer a1 = new WhitespaceAnalyzer();
[ecj-lint] ^^
[ecj-lint] Resource leak: 'a1' is never closed
[ecj-lint] ----------
[ecj-lint] 8. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/highlight/HighlighterTest.java (at line 204)
[ecj-lint] OffsetWindowTokenFilter tots = new OffsetWindowTokenFilter(tokenStream);
[ecj-lint] ^^^^
[ecj-lint] Resource leak: 'tots' is never closed
[ecj-lint] ----------
[ecj-lint] 9. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/highlight/HighlighterTest.java (at line 208)
[ecj-lint] Analyzer a2 = new WhitespaceAnalyzer();
[ecj-lint] ^^
[ecj-lint] Resource leak: 'a2' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 10. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/response/TestBinaryResponseWriter.java (at line 63)
[ecj-lint] NamedList res = (NamedList) new JavaBinCodec().unmarshal(new ByteArrayInputStream(baos.toByteArray()));
[ecj-lint] ^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 11. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/response/TestJavabinTupleStreamParser.java (at line 72)
[ecj-lint] JavabinTupleStreamParser parser = new JavabinTupleStreamParser(new ByteArrayInputStream(bytes), true);
[ecj-lint] ^^^^^^
[ecj-lint] Resource leak: 'parser' is never closed
[ecj-lint] ----------
[ecj-lint] 12. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/response/TestJavabinTupleStreamParser.java (at line 173)
[ecj-lint] new JavaBinCodec().marshal(response.getValues(), baos);
[ecj-lint] ^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] 13. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/response/TestJavabinTupleStreamParser.java (at line 175)
[ecj-lint] Object o = new JavaBinCodec().unmarshal(new ByteArrayInputStream(bytes));
[ecj-lint] ^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] 14. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/response/TestJavabinTupleStreamParser.java (at line 178)
[ecj-lint] JavabinTupleStreamParser parser = new JavabinTupleStreamParser(new ByteArrayInputStream(bytes), false);
[ecj-lint] ^^^^^^
[ecj-lint] Resource leak: 'parser' is never closed
[ecj-lint] ----------
[ecj-lint] 15. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/response/TestJavabinTupleStreamParser.java (at line 192)
[ecj-lint] new JavaBinCodec().marshal(response.getValues(), baos);
[ecj-lint] ^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 16. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/response/TestPushWriter.java (at line 57)
[ecj-lint] m = (Map) new JavaBinCodec().unmarshal(new ByteArrayInputStream(baos.toByteArray()));
[ecj-lint] ^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 17. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/response/transform/TestSubQueryTransformer.java (at line 387)
[ecj-lint] unmarshalled = (NamedList<Object>) new JavaBinCodec().unmarshal(
[ecj-lint] ^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 18. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/search/TestDocSet.java (at line 241)
[ecj-lint] return loadfactor!=0 ? new HashDocSet(a,0,n,1/loadfactor) : new HashDocSet(a,0,n);
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] 19. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/search/TestDocSet.java (at line 526)
[ecj-lint] DocSet a = new BitDocSet(bs);
[ecj-lint] ^
[ecj-lint] Resource leak: 'a' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 20. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/security/TestPKIAuthenticationPlugin.java (at line 85)
[ecj-lint] final MockPKIAuthenticationPlugin mock = new MockPKIAuthenticationPlugin(null, nodeName);
[ecj-lint] ^^^^
[ecj-lint] Resource leak: 'mock' is never closed
[ecj-lint] ----------
[ecj-lint] 21. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/security/TestPKIAuthenticationPlugin.java (at line 138)
[ecj-lint] MockPKIAuthenticationPlugin mock1 = new MockPKIAuthenticationPlugin(null, nodeName) {
[ecj-lint] ^^^^^
[ecj-lint] Resource leak: 'mock1' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 22. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/security/TestRuleBasedAuthorizationPlugin.java (at line 365)
[ecj-lint] RuleBasedAuthorizationPlugin plugin = new RuleBasedAuthorizationPlugin();
[ecj-lint] ^^^^^^
[ecj-lint] Resource leak: 'plugin' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 23. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/security/TestSha256AuthenticationProvider.java (at line 49)
[ecj-lint] BasicAuthPlugin basicAuthPlugin = new BasicAuthPlugin();
[ecj-lint] ^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'basicAuthPlugin' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 24. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/spelling/SimpleQueryConverter.java (at line 42)
[ecj-lint] WhitespaceAnalyzer analyzer = new WhitespaceAnalyzer();
[ecj-lint] ^^^^^^^^
[ecj-lint] Resource leak: 'analyzer' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 25. ERROR in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/store/blockcache/BlockCacheTest.java (at line 22)
[ecj-lint] import java.util.concurrent.atomic.AtomicInteger;
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] The import java.util.concurrent.atomic.AtomicInteger is never used
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 26. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/uninverting/TestFieldCacheVsDocValues.java (at line 139)
[ecj-lint] IndexWriter w = new IndexWriter(d, newIndexWriterConfig(analyzer));
[ecj-lint] ^
[ecj-lint] Resource leak: 'w' is never closed
[ecj-lint] ----------
[ecj-lint] 27. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/uninverting/TestFieldCacheVsDocValues.java (at line 172)
[ecj-lint] throw iae;
[ecj-lint] ^^^^^^^^^^
[ecj-lint] Resource leak: 'w' is not closed at this location
[ecj-lint] ----------
[ecj-lint] 28. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/uninverting/TestFieldCacheVsDocValues.java (at line 178)
[ecj-lint] return;
[ecj-lint] ^^^^^^^
[ecj-lint] Resource leak: 'w' is not closed at this location
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 29. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/update/AddBlockUpdateTest.java (at line 415)
[ecj-lint] new JavaBinCodec().marshal(topDocument, os);
[ecj-lint] ^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] 30. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/update/AddBlockUpdateTest.java (at line 419)
[ecj-lint] SolrInputDocument result = (SolrInputDocument) new JavaBinCodec().unmarshal(is);
[ecj-lint] ^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 31. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/update/SolrIndexSplitterTest.java (at line 181)
[ecj-lint] EmbeddedSolrServer server1 = new EmbeddedSolrServer(h.getCoreContainer(), "split1");
[ecj-lint] ^^^^^^^
[ecj-lint] Resource leak: 'server1' is never closed
[ecj-lint] ----------
[ecj-lint] 32. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test/org/apache/solr/update/SolrIndexSplitterTest.java (at line 182)
[ecj-lint] EmbeddedSolrServer server2 = new EmbeddedSolrServer(h.getCoreContainer(), "split2");
[ecj-lint] ^^^^^^^
[ecj-lint] Resource leak: 'server2' is never closed
[ecj-lint] ----------
[ecj-lint] 32 problems (1 error, 31 warnings)
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/build.xml:775: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/build.xml:101: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build.xml:674: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/common-build.xml:2010: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/common-build.xml:2043: Compile failed; see the compiler error output for details.
Total time: 116 minutes 49 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3823
- Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3823/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC
1 tests failed.
FAILED: org.apache.solr.cloud.PeerSyncReplicationTest.test
Error Message:
timeout waiting to see all nodes active
Stack Trace:
java.lang.AssertionError: timeout waiting to see all nodes active
at __randomizedtesting.SeedInfo.seed([6BA1B11E3B0653F4:E3F58EC495FA3E0C]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.PeerSyncReplicationTest.waitTillNodesActive(PeerSyncReplicationTest.java:326)
at org.apache.solr.cloud.PeerSyncReplicationTest.bringUpDeadNodeAndEnsureNoReplication(PeerSyncReplicationTest.java:277)
at org.apache.solr.cloud.PeerSyncReplicationTest.forceNodeFailureAndDoPeerSync(PeerSyncReplicationTest.java:259)
at org.apache.solr.cloud.PeerSyncReplicationTest.test(PeerSyncReplicationTest.java:138)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 12377 lines...]
[junit4] Suite: org.apache.solr.cloud.PeerSyncReplicationTest
[junit4] 2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/init-core-data-001
[junit4] 2> 2878755 INFO (SUITE-PeerSyncReplicationTest-seed#[6BA1B11E3B0653F4]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields
[junit4] 2> 2878759 INFO (SUITE-PeerSyncReplicationTest-seed#[6BA1B11E3B0653F4]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN) w/ MAC_OS_X supressed clientAuth
[junit4] 2> 2878759 INFO (SUITE-PeerSyncReplicationTest-seed#[6BA1B11E3B0653F4]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 2878763 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2878763 INFO (Thread-3611) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2878763 INFO (Thread-3611) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2878873 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.ZkTestServer start zk server on port:52265
[junit4] 2> 2878908 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 2878914 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 2878920 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2878924 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 2878928 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 2878933 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 2878936 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 2878940 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 2878944 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 2878949 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 2878953 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 2881847 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/control-001/cores/collection1
[junit4] 2> 2881849 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2881851 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@640a566d{/,null,AVAILABLE}
[junit4] 2> 2881853 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@17ca7da6{HTTP/1.1,[http/1.1]}{127.0.0.1:52268}
[junit4] 2> 2881854 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.e.j.s.Server Started @2887141ms
[junit4] 2> 2881854 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/tempDir-001/control/data, hostContext=/, hostPort=52268, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/control-001/cores}
[junit4] 2> 2881854 ERROR (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2881855 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 2881855 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2881855 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2881855 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-10T09:22:59.796Z
[junit4] 2> 2881859 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2881859 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/control-001/solr.xml
[junit4] 2> 2881880 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52265/solr
[junit4] 2> 2881925 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [n:127.0.0.1:52268_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:52268_
[junit4] 2> 2881927 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [n:127.0.0.1:52268_ ] o.a.s.c.Overseer Overseer (id=97433588643069956-127.0.0.1:52268_-n_0000000000) starting
[junit4] 2> 2881941 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [n:127.0.0.1:52268_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52268_
[junit4] 2> 2881945 INFO (zkCallback-3305-thread-1-processing-n:127.0.0.1:52268_) [n:127.0.0.1:52268_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2882044 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [n:127.0.0.1:52268_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/control-001/cores
[junit4] 2> 2882044 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [n:127.0.0.1:52268_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2882048 INFO (OverseerStateUpdate-97433588643069956-127.0.0.1:52268_-n_0000000000) [n:127.0.0.1:52268_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 2883074 INFO (coreLoadExecutor-8026-thread-1-processing-n:127.0.0.1:52268_) [n:127.0.0.1:52268_ c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2883087 INFO (coreLoadExecutor-8026-thread-1-processing-n:127.0.0.1:52268_) [n:127.0.0.1:52268_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2883182 WARN (coreLoadExecutor-8026-thread-1-processing-n:127.0.0.1:52268_) [n:127.0.0.1:52268_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2883184 INFO (coreLoadExecutor-8026-thread-1-processing-n:127.0.0.1:52268_) [n:127.0.0.1:52268_ c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 2883205 INFO (coreLoadExecutor-8026-thread-1-processing-n:127.0.0.1:52268_) [n:127.0.0.1:52268_ c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 2883205 INFO (coreLoadExecutor-8026-thread-1-processing-n:127.0.0.1:52268_) [n:127.0.0.1:52268_ c:control_collection x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/control-001/cores/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/control-001/cores/collection1/data/]
[junit4] 2> 2883205 INFO (coreLoadExecutor-8026-thread-1-processing-n:127.0.0.1:52268_) [n:127.0.0.1:52268_ c:control_collection x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@38e6b9e1
[junit4] 2> 2883208 INFO (coreLoadExecutor-8026-thread-1-processing-n:127.0.0.1:52268_) [n:127.0.0.1:52268_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=12, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 2883227 WARN (coreLoadExecutor-8026-thread-1-processing-n:127.0.0.1:52268_) [n:127.0.0.1:52268_ c:control_collection x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2883284 INFO (coreLoadExecutor-8026-thread-1-processing-n:127.0.0.1:52268_) [n:127.0.0.1:52268_ c:control_collection x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2883285 INFO (coreLoadExecutor-8026-thread-1-processing-n:127.0.0.1:52268_) [n:127.0.0.1:52268_ c:control_collection x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2883287 INFO (coreLoadExecutor-8026-thread-1-processing-n:127.0.0.1:52268_) [n:127.0.0.1:52268_ c:control_collection x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2883287 INFO (coreLoadExecutor-8026-thread-1-processing-n:127.0.0.1:52268_) [n:127.0.0.1:52268_ c:control_collection x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2883287 INFO (coreLoadExecutor-8026-thread-1-processing-n:127.0.0.1:52268_) [n:127.0.0.1:52268_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=852295990, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 2883288 INFO (coreLoadExecutor-8026-thread-1-processing-n:127.0.0.1:52268_) [n:127.0.0.1:52268_ c:control_collection x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@14ca0187[collection1] main]
[junit4] 2> 2883290 INFO (coreLoadExecutor-8026-thread-1-processing-n:127.0.0.1:52268_) [n:127.0.0.1:52268_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2883291 INFO (coreLoadExecutor-8026-thread-1-processing-n:127.0.0.1:52268_) [n:127.0.0.1:52268_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2883291 INFO (coreLoadExecutor-8026-thread-1-processing-n:127.0.0.1:52268_) [n:127.0.0.1:52268_ c:control_collection x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2883292 INFO (searcherExecutor-8027-thread-1-processing-n:127.0.0.1:52268_ x:collection1 c:control_collection) [n:127.0.0.1:52268_ c:control_collection x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@14ca0187[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2883292 INFO (coreLoadExecutor-8026-thread-1-processing-n:127.0.0.1:52268_) [n:127.0.0.1:52268_ c:control_collection x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558937423034974208
[junit4] 2> 2883304 INFO (coreZkRegister-8019-thread-1-processing-n:127.0.0.1:52268_ x:collection1 c:control_collection) [n:127.0.0.1:52268_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2883304 INFO (coreZkRegister-8019-thread-1-processing-n:127.0.0.1:52268_ x:collection1 c:control_collection) [n:127.0.0.1:52268_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2883304 INFO (coreZkRegister-8019-thread-1-processing-n:127.0.0.1:52268_ x:collection1 c:control_collection) [n:127.0.0.1:52268_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:52268/collection1/
[junit4] 2> 2883304 INFO (coreZkRegister-8019-thread-1-processing-n:127.0.0.1:52268_ x:collection1 c:control_collection) [n:127.0.0.1:52268_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2883304 INFO (coreZkRegister-8019-thread-1-processing-n:127.0.0.1:52268_ x:collection1 c:control_collection) [n:127.0.0.1:52268_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:52268/collection1/ has no replicas
[junit4] 2> 2883312 INFO (coreZkRegister-8019-thread-1-processing-n:127.0.0.1:52268_ x:collection1 c:control_collection) [n:127.0.0.1:52268_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:52268/collection1/ shard1
[junit4] 2> 2883477 INFO (coreZkRegister-8019-thread-1-processing-n:127.0.0.1:52268_ x:collection1 c:control_collection) [n:127.0.0.1:52268_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2883793 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2883795 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:52265/solr ready
[junit4] 2> 2883795 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 2883795 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
[junit4] 2> 2884317 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/shard-1-001/cores/collection1
[junit4] 2> 2884318 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/shard-1-001
[junit4] 2> 2884319 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2884321 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@58c181bc{/,null,AVAILABLE}
[junit4] 2> 2884321 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5a8e1820{HTTP/1.1,[http/1.1]}{127.0.0.1:52274}
[junit4] 2> 2884321 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.e.j.s.Server Started @2889608ms
[junit4] 2> 2884322 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=52274, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/shard-1-001/cores}
[junit4] 2> 2884322 ERROR (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2884323 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 2884323 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2884323 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2884323 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-10T09:23:02.264Z
[junit4] 2> 2884327 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2884327 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/shard-1-001/solr.xml
[junit4] 2> 2884414 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52265/solr
[junit4] 2> 2884434 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [n:127.0.0.1:52274_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2884444 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [n:127.0.0.1:52274_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52274_
[junit4] 2> 2884449 INFO (zkCallback-3315-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2884450 INFO (zkCallback-3305-thread-1-processing-n:127.0.0.1:52268_) [n:127.0.0.1:52268_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2884452 INFO (zkCallback-3309-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2884528 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [n:127.0.0.1:52274_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/shard-1-001/cores
[junit4] 2> 2884528 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [n:127.0.0.1:52274_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2884533 INFO (OverseerStateUpdate-97433588643069956-127.0.0.1:52268_-n_0000000000) [n:127.0.0.1:52268_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 2884644 INFO (zkCallback-3315-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2885555 INFO (coreLoadExecutor-8037-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2885574 INFO (coreLoadExecutor-8037-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2885695 WARN (coreLoadExecutor-8037-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2885697 INFO (coreLoadExecutor-8037-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 2885720 INFO (coreLoadExecutor-8037-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 2885720 INFO (coreLoadExecutor-8037-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/shard-1-001/cores/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/shard-1-001/cores/collection1/data/]
[junit4] 2> 2885720 INFO (coreLoadExecutor-8037-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@38e6b9e1
[junit4] 2> 2885724 INFO (coreLoadExecutor-8037-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=12, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 2885747 WARN (coreLoadExecutor-8037-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2885832 INFO (coreLoadExecutor-8037-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2885832 INFO (coreLoadExecutor-8037-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2885835 INFO (coreLoadExecutor-8037-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2885835 INFO (coreLoadExecutor-8037-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2885836 INFO (coreLoadExecutor-8037-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=852295990, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 2885837 INFO (coreLoadExecutor-8037-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5c49823f[collection1] main]
[junit4] 2> 2885839 INFO (coreLoadExecutor-8037-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2885840 INFO (coreLoadExecutor-8037-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2885840 INFO (coreLoadExecutor-8037-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2885841 INFO (searcherExecutor-8038-thread-1-processing-n:127.0.0.1:52274_ x:collection1 c:collection1) [n:127.0.0.1:52274_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5c49823f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2885843 INFO (coreLoadExecutor-8037-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558937425709891584
[junit4] 2> 2885854 INFO (coreZkRegister-8032-thread-1-processing-n:127.0.0.1:52274_ x:collection1 c:collection1) [n:127.0.0.1:52274_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2885854 INFO (coreZkRegister-8032-thread-1-processing-n:127.0.0.1:52274_ x:collection1 c:collection1) [n:127.0.0.1:52274_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2885854 INFO (coreZkRegister-8032-thread-1-processing-n:127.0.0.1:52274_ x:collection1 c:collection1) [n:127.0.0.1:52274_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:52274/collection1/
[junit4] 2> 2885854 INFO (coreZkRegister-8032-thread-1-processing-n:127.0.0.1:52274_ x:collection1 c:collection1) [n:127.0.0.1:52274_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2885854 INFO (coreZkRegister-8032-thread-1-processing-n:127.0.0.1:52274_ x:collection1 c:collection1) [n:127.0.0.1:52274_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:52274/collection1/ has no replicas
[junit4] 2> 2885862 INFO (coreZkRegister-8032-thread-1-processing-n:127.0.0.1:52274_ x:collection1 c:collection1) [n:127.0.0.1:52274_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:52274/collection1/ shard1
[junit4] 2> 2885968 INFO (zkCallback-3315-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2886015 INFO (coreZkRegister-8032-thread-1-processing-n:127.0.0.1:52274_ x:collection1 c:collection1) [n:127.0.0.1:52274_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2886126 INFO (zkCallback-3315-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2886962 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/shard-2-001/cores/collection1
[junit4] 2> 2886964 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/shard-2-001
[junit4] 2> 2886964 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2886967 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@55fbf639{/,null,AVAILABLE}
[junit4] 2> 2886971 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@397b72e7{HTTP/1.1,[http/1.1]}{127.0.0.1:52304}
[junit4] 2> 2886972 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.e.j.s.Server Started @2892259ms
[junit4] 2> 2886972 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/, hostPort=52304, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/shard-2-001/cores}
[junit4] 2> 2886972 ERROR (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2886973 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 2886974 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2886975 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2886975 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-10T09:23:04.916Z
[junit4] 2> 2886990 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2886990 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/shard-2-001/solr.xml
[junit4] 2> 2887002 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52265/solr
[junit4] 2> 2887029 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [n:127.0.0.1:52304_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2887046 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [n:127.0.0.1:52304_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52304_
[junit4] 2> 2887049 INFO (zkCallback-3315-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2887049 INFO (zkCallback-3305-thread-1-processing-n:127.0.0.1:52268_) [n:127.0.0.1:52268_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2887049 INFO (zkCallback-3309-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2887050 INFO (zkCallback-3321-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2887164 INFO (zkCallback-3315-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2887364 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [n:127.0.0.1:52304_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/shard-2-001/cores
[junit4] 2> 2887364 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [n:127.0.0.1:52304_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2887369 INFO (OverseerStateUpdate-97433588643069956-127.0.0.1:52268_-n_0000000000) [n:127.0.0.1:52268_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 2887476 INFO (zkCallback-3315-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2887476 INFO (zkCallback-3321-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2888391 INFO (coreLoadExecutor-8048-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2888407 INFO (coreLoadExecutor-8048-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2888532 WARN (coreLoadExecutor-8048-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2888535 INFO (coreLoadExecutor-8048-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 2888557 INFO (coreLoadExecutor-8048-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 2888557 INFO (coreLoadExecutor-8048-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/shard-2-001/cores/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/shard-2-001/cores/collection1/data/]
[junit4] 2> 2888557 INFO (coreLoadExecutor-8048-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@38e6b9e1
[junit4] 2> 2888572 INFO (coreLoadExecutor-8048-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=12, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 2888582 WARN (coreLoadExecutor-8048-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2888668 INFO (coreLoadExecutor-8048-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2888668 INFO (coreLoadExecutor-8048-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2888671 INFO (coreLoadExecutor-8048-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2888671 INFO (coreLoadExecutor-8048-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2888671 INFO (coreLoadExecutor-8048-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=852295990, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 2888672 INFO (coreLoadExecutor-8048-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5c92562d[collection1] main]
[junit4] 2> 2888674 INFO (coreLoadExecutor-8048-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2888675 INFO (coreLoadExecutor-8048-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2888675 INFO (coreLoadExecutor-8048-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2888677 INFO (searcherExecutor-8049-thread-1-processing-n:127.0.0.1:52304_ x:collection1 c:collection1) [n:127.0.0.1:52304_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5c92562d[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2888677 INFO (coreLoadExecutor-8048-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558937428681555968
[junit4] 2> 2888683 INFO (coreZkRegister-8043-thread-1-processing-n:127.0.0.1:52304_ x:collection1 c:collection1) [n:127.0.0.1:52304_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkController Core needs to recover:collection1
[junit4] 2> 2888684 INFO (updateExecutor-3318-thread-1-processing-n:127.0.0.1:52304_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:52304_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 2888684 INFO (recoveryExecutor-3319-thread-1-processing-n:127.0.0.1:52304_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:52304_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 2888684 INFO (recoveryExecutor-3319-thread-1-processing-n:127.0.0.1:52304_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:52304_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 2888684 INFO (recoveryExecutor-3319-thread-1-processing-n:127.0.0.1:52304_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:52304_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1]
[junit4] 2> 2888684 INFO (recoveryExecutor-3319-thread-1-processing-n:127.0.0.1:52304_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:52304_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 2888684 INFO (recoveryExecutor-3319-thread-1-processing-n:127.0.0.1:52304_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:52304_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core [collection1] as recovering, leader is [http://127.0.0.1:52274/collection1/] and I am [http://127.0.0.1:52304/collection1/]
[junit4] 2> 2888693 INFO (recoveryExecutor-3319-thread-1-processing-n:127.0.0.1:52304_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:52304_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:52274]; [WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:52304_&coreNodeName=core_node2&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 2888696 INFO (qtp1646759755-17774) [n:127.0.0.1:52274_ ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 2888697 INFO (qtp1646759755-17774) [n:127.0.0.1:52274_ ] o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 (shard1 of collection1) have state: recovering
[junit4] 2> 2888697 INFO (qtp1646759755-17774) [n:127.0.0.1:52274_ ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:52304_, coreNodeName=core_node2, onlyIfActiveCheckResult=false, nodeProps: core_node2:{"core":"collection1","base_url":"http://127.0.0.1:52304","node_name":"127.0.0.1:52304_","state":"down"}
[junit4] 2> 2888796 INFO (zkCallback-3315-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2888796 INFO (zkCallback-3321-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2889460 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/shard-3-001/cores/collection1
[junit4] 2> 2889462 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/shard-3-001
[junit4] 2> 2889462 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2889464 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@75491ff7{/,null,AVAILABLE}
[junit4] 2> 2889465 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2584af50{HTTP/1.1,[http/1.1]}{127.0.0.1:52309}
[junit4] 2> 2889465 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.e.j.s.Server Started @2894752ms
[junit4] 2> 2889465 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/tempDir-001/jetty3, solrconfig=solrconfig.xml, hostContext=/, hostPort=52309, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/shard-3-001/cores}
[junit4] 2> 2889466 ERROR (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2889466 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 2889466 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2889467 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2889467 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-10T09:23:07.408Z
[junit4] 2> 2889471 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2889471 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/shard-3-001/solr.xml
[junit4] 2> 2889483 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52265/solr
[junit4] 2> 2889501 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [n:127.0.0.1:52309_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2889510 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [n:127.0.0.1:52309_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52309_
[junit4] 2> 2889515 INFO (zkCallback-3315-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2889515 INFO (zkCallback-3305-thread-2-processing-n:127.0.0.1:52268_) [n:127.0.0.1:52268_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2889516 INFO (zkCallback-3328-thread-1-processing-n:127.0.0.1:52309_) [n:127.0.0.1:52309_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2889517 INFO (zkCallback-3309-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2889517 INFO (zkCallback-3321-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2889631 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [n:127.0.0.1:52309_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/shard-3-001/cores
[junit4] 2> 2889631 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [n:127.0.0.1:52309_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2889633 INFO (zkCallback-3315-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2889633 INFO (zkCallback-3321-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2889637 INFO (OverseerStateUpdate-97433588643069956-127.0.0.1:52268_-n_0000000000) [n:127.0.0.1:52268_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 2889702 INFO (qtp1646759755-17774) [n:127.0.0.1:52274_ ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:52304_, coreNodeName=core_node2, onlyIfActiveCheckResult=false, nodeProps: core_node2:{"core":"collection1","base_url":"http://127.0.0.1:52304","node_name":"127.0.0.1:52304_","state":"recovering"}
[junit4] 2> 2889702 INFO (qtp1646759755-17774) [n:127.0.0.1:52274_ ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 2889702 INFO (qtp1646759755-17774) [n:127.0.0.1:52274_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:52304_&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node2&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1005
[junit4] 2> 2889744 INFO (zkCallback-3321-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2889744 INFO (zkCallback-3315-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2889744 INFO (zkCallback-3328-thread-1-processing-n:127.0.0.1:52309_) [n:127.0.0.1:52309_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2890658 INFO (coreLoadExecutor-8059-thread-1-processing-n:127.0.0.1:52309_) [n:127.0.0.1:52309_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2890707 INFO (coreLoadExecutor-8059-thread-1-processing-n:127.0.0.1:52309_) [n:127.0.0.1:52309_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2890807 WARN (coreLoadExecutor-8059-thread-1-processing-n:127.0.0.1:52309_) [n:127.0.0.1:52309_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2890809 INFO (coreLoadExecutor-8059-thread-1-processing-n:127.0.0.1:52309_) [n:127.0.0.1:52309_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 2890833 INFO (coreLoadExecutor-8059-thread-1-processing-n:127.0.0.1:52309_) [n:127.0.0.1:52309_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 2890833 INFO (coreLoadExecutor-8059-thread-1-processing-n:127.0.0.1:52309_) [n:127.0.0.1:52309_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/shard-3-001/cores/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_6BA1B11E3B0653F4-001/shard-3-001/cores/collection1/data/]
[junit4] 2> 2890833 INFO (coreLoadExecutor-8059-thread-1-processing-n:127.0.0.1:52309_) [n:127.0.0.1:52309_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@38e6b9e1
[junit4] 2> 2890838 INFO (coreLoadExecutor-8059-thread-1-processing-n:127.0.0.1:52309_) [n:127.0.0.1:52309_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=12, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 2890854 WARN (coreLoadExecutor-8059-thread-1-processing-n:127.0.0.1:52309_) [n:127.0.0.1:52309_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2890935 INFO (coreLoadExecutor-8059-thread-1-processing-n:127.0.0.1:52309_) [n:127.0.0.1:52309_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2890936 INFO (coreLoadExecutor-8059-thread-1-processing-n:127.0.0.1:52309_) [n:127.0.0.1:52309_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2890938 INFO (coreLoadExecutor-8059-thread-1-processing-n:127.0.0.1:52309_) [n:127.0.0.1:52309_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2890938 INFO (coreLoadExecutor-8059-thread-1-processing-n:127.0.0.1:52309_) [n:127.0.0.1:52309_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2890939 INFO (coreLoadExecutor-8059-thread-1-processing-n:127.0.0.1:52309_) [n:127.0.0.1:52309_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=852295990, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 2890945 INFO (coreLoadExecutor-8059-thread-1-processing-n:127.0.0.1:52309_) [n:127.0.0.1:52309_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@50a9ee1e[collection1] main]
[junit4] 2> 2890946 INFO (coreLoadExecutor-8059-thread-1-processing-n:127.0.0.1:52309_) [n:127.0.0.1:52309_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2890947 INFO (coreLoadExecutor-8059-thread-1-processing-n:127.0.0.1:52309_) [n:127.0.0.1:52309_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2890947 INFO (coreLoadExecutor-8059-thread-1-processing-n:127.0.0.1:52309_) [n:127.0.0.1:52309_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2890949 INFO (searcherExecutor-8060-thread-1-processing-n:127.0.0.1:52309_ x:collection1 c:collection1) [n:127.0.0.1:52309_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@50a9ee1e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2890950 INFO (coreLoadExecutor-8059-thread-1-processing-n:127.0.0.1:52309_) [n:127.0.0.1:52309_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558937431064969216
[junit4] 2> 2890957 INFO (coreZkRegister-8054-thread-1-processing-n:127.0.0.1:52309_ x:collection1 c:collection1) [n:127.0.0.1:52309_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.ZkController Core needs to recover:collection1
[junit4] 2> 2890958 INFO (updateExecutor-3325-thread-1-processing-n:127.0.0.1:52309_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:52309_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 2890958 INFO (recoveryExecutor-3326-thread-1-processing-n:127.0.0.1:52309_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:52309_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 2890958 INFO (recoveryExecutor-3326-thread-1-processing-n:127.0.0.1:52309_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:52309_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 2890958 INFO (recoveryExecutor-3326-thread-1-processing-n:127.0.0.1:52309_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:52309_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1]
[junit4] 2> 2890958 INFO (recoveryExecutor-3326-thread-1-processing-n:127.0.0.1:52309_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:52309_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 2890959 INFO (recoveryExecutor-3326-thread-1-processing-n:127.0.0.1:52309_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:52309_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core [collection1] as recovering, leader is [http://127.0.0.1:52274/collection1/] and I am [http://127.0.0.1:52309/collection1/]
[junit4] 2> 2890964 INFO (recoveryExecutor-3326-thread-1-processing-n:127.0.0.1:52309_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:52309_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:52274]; [WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:52309_&coreNodeName=core_node3&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 2890966 INFO (qtp1646759755-17778) [n:127.0.0.1:52274_ ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 2890967 INFO (qtp1646759755-17778) [n:127.0.0.1:52274_ ] o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 (shard1 of collection1) have state: recovering
[junit4] 2> 2890967 INFO (qtp1646759755-17778) [n:127.0.0.1:52274_ ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:52309_, coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: core_node3:{"core":"collection1","base_url":"http://127.0.0.1:52309","node_name":"127.0.0.1:52309_","state":"down"}
[junit4] 2> 2891070 INFO (zkCallback-3321-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2891070 INFO (zkCallback-3315-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2891070 INFO (zkCallback-3328-thread-1-processing-n:127.0.0.1:52309_) [n:127.0.0.1:52309_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2891406 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 2891406 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Wait for recoveries to finish - wait 30 for each attempt
[junit4] 2> 2891406 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):30
[junit4] 2> 2891967 INFO (qtp1646759755-17778) [n:127.0.0.1:52274_ ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:52309_, coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: core_node3:{"core":"collection1","base_url":"http://127.0.0.1:52309","node_name":"127.0.0.1:52309_","state":"recovering"}
[junit4] 2> 2891967 INFO (qtp1646759755-17778) [n:127.0.0.1:52274_ ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 2891967 INFO (qtp1646759755-17778) [n:127.0.0.1:52274_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:52309_&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node3&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1002
[junit4] 2> 2896706 INFO (recoveryExecutor-3319-thread-1-processing-n:127.0.0.1:52304_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:52304_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:52274/collection1/] - recoveringAfterStartup=[true]
[junit4] 2> 2896706 INFO (recoveryExecutor-3319-thread-1-processing-n:127.0.0.1:52304_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:52304_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.PeerSync PeerSync: core=collection1 url=http://127.0.0.1:52304 START replicas=[http://127.0.0.1:52274/collection1/] nUpdates=1000
[junit4] 2> 2896709 INFO (qtp1646759755-17775) [n:127.0.0.1:52274_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 2896709 INFO (qtp1646759755-17775) [n:127.0.0.1:52274_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/get params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 2896710 INFO (recoveryExecutor-3319-thread-1-processing-n:127.0.0.1:52304_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:52304_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 2896710 INFO (recoveryExecutor-3319-thread-1-processing-n:127.0.0.1:52304_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:52304_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.PeerSync We are already in sync. No need to do a PeerSync
[junit4] 2> 2896710 INFO (recoveryExecutor-3319-thread-1-processing-n:127.0.0.1:52304_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:52304_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2896710 INFO (recoveryExecutor-3319-thread-1-processing-n:127.0.0.1:52304_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:52304_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 2896710 INFO (recoveryExecutor-3319-thread-1-processing-n:127.0.0.1:52304_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:52304_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2896711 INFO (recoveryExecutor-3319-thread-1-processing-n:127.0.0.1:52304_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:52304_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy PeerSync stage of recovery was successful.
[junit4] 2> 2896711 INFO (recoveryExecutor-3319-thread-1-processing-n:127.0.0.1:52304_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:52304_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Replaying updates buffered during PeerSync.
[junit4] 2> 2896711 INFO (recoveryExecutor-3319-thread-1-processing-n:127.0.0.1:52304_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:52304_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy No replay needed.
[junit4] 2> 2896711 INFO (recoveryExecutor-3319-thread-1-processing-n:127.0.0.1:52304_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:52304_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Registering as Active after recovery.
[junit4] 2> 2896717 INFO (zkCallback-3321-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2896717 INFO (zkCallback-3328-thread-1-processing-n:127.0.0.1:52309_) [n:127.0.0.1:52309_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2896717 INFO (zkCallback-3315-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2898977 INFO (recoveryExecutor-3326-thread-1-processing-n:127.0.0.1:52309_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:52309_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:52274/collection1/] - recoveringAfterStartup=[true]
[junit4] 2> 2898978 INFO (recoveryExecutor-3326-thread-1-processing-n:127.0.0.1:52309_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:52309_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.PeerSync PeerSync: core=collection1 url=http://127.0.0.1:52309 START replicas=[http://127.0.0.1:52274/collection1/] nUpdates=1000
[junit4] 2> 2898981 INFO (qtp1646759755-17777) [n:127.0.0.1:52274_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 2898981 INFO (qtp1646759755-17777) [n:127.0.0.1:52274_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/get params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2898982 INFO (recoveryExecutor-3326-thread-1-processing-n:127.0.0.1:52309_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:52309_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 2898982 INFO (recoveryExecutor-3326-thread-1-processing-n:127.0.0.1:52309_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:52309_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.PeerSync We are already in sync. No need to do a PeerSync
[junit4] 2> 2898982 INFO (recoveryExecutor-3326-thread-1-processing-n:127.0.0.1:52309_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:52309_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2898983 INFO (recoveryExecutor-3326-thread-1-processing-n:127.0.0.1:52309_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:52309_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 2898983 INFO (recoveryExecutor-3326-thread-1-processing-n:127.0.0.1:52309_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:52309_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2898983 INFO (recoveryExecutor-3326-thread-1-processing-n:127.0.0.1:52309_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:52309_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy PeerSync stage of recovery was successful.
[junit4] 2> 2898983 INFO (recoveryExecutor-3326-thread-1-processing-n:127.0.0.1:52309_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:52309_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Replaying updates buffered during PeerSync.
[junit4] 2> 2898983 INFO (recoveryExecutor-3326-thread-1-processing-n:127.0.0.1:52309_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:52309_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy No replay needed.
[junit4] 2> 2898984 INFO (recoveryExecutor-3326-thread-1-processing-n:127.0.0.1:52309_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:52309_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Registering as Active after recovery.
[junit4] 2> 2898989 INFO (zkCallback-3315-thread-1-processing-n:127.0.0.1:52274_) [n:127.0.0.1:52274_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2898989 INFO (zkCallback-3321-thread-1-processing-n:127.0.0.1:52304_) [n:127.0.0.1:52304_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2898989 INFO (zkCallback-3328-thread-1-processing-n:127.0.0.1:52309_) [n:127.0.0.1:52309_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2899465 INFO (TEST-PeerSyncReplicationTest.test-seed#[6BA1B11E3B0653F4]) [ ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: collection1
[junit4] 2> 2899467 INFO (qtp261912456-17734) [n:127.0.0.1:52268_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2899467 INFO (qtp261912456-17734) [n:127.0.0.1:52268_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 2899469 INFO (qtp261912456-17734) [n:127.0.0.1:52268_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2899469 INFO (qtp261912456-17734) [n:127.0.0.1:52268_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 1
[junit4] 2> 2899473 INFO (qtp1646759755-17779) [n:127.0.0.1:52274_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2899473 INFO (qtp1646759755-17779) [n:127.0.0.1:52274_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 2899476 INFO (qtp552230015-17803) [n:127.0.0.1:52304_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2899476 INFO (qtp552230015-17803) [n:127.0.0.1:52304_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 2899476 INFO (qtp707322001-17838) [n:127.0.0.1:52309_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2899476 INFO (qtp1646759755-17779) [n:127.0.0.1:52274_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2899476 INFO (qtp707322001-17838) [n:127.0.0.1:52309_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 2899476 INFO (qtp552230015-17803) [n:127.0.0.1:52304_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2899476 INFO (qtp1646759755-17779) [n:127.0.0.1:52274_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:52274/collection1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 3
[junit4] 2> 2899476 INFO (qtp552230015-17803) [n:127.0.0.1:52304_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:52274/collection1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 0
[junit4] 2> 2899477 INFO (qtp707322001-17838) [n:127.0.0.1:52309_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2899477 INFO (qtp707322001-17838) [n:127.0.0.1:52309_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:52274/collection1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 1
[junit4] 2> 2899477 INFO (qtp1646759755-17778) [n:127.0.0.1:52274_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={_stateVer_=collection1:11&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 6
[junit4] 2> 2899481 INFO (qtp1646759755-17772) [n:127.0.0.1:52274_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&distrib=false&tests=checkShardConsistency&rows=0&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 2899482 INFO (qtp552230015-17805) [n:127.0.0.1:52304_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&distrib=false&tests=checkShardConsistency&rows=0&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 2899483 INFO (qtp707322001-17840) [n:127.0.0.1:52309_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&distrib=false&tests=checkShardConsistency&rows=0&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 2901492 INFO (qtp261912456-17735) [n:127.0.0.1:52268_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{deleteByQuery=*:* (-1558937442114863104)} 0 4
[junit4] 2> 2901499 INFO (qtp552230015-17806) [n:127.0.0.1:52304_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1558937442120105984&distrib.from=http://127.0.0.1:52274/collection1/&wt=javabin&version=2}{deleteByQuery=*:* (-1558937442120105984)} 0 3
[junit4] 2> 2901500 INFO (qtp707322001-17841) [n:127.0.0.1:52309_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1558937442120105984&distrib.from=http://127.0.0.1:52274/collection1/&wt=javabin&version=2}{deleteByQuery=*:* (-1558937442120105984)} 0 4
[junit4] 2> 2901501 INFO (qtp1646759755-17775) [n:127.0.0.1:52274_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{deleteByQuery=*:* (-1558937442120105984)} 0 8
[junit4] 2> 2901506 INFO (qtp552230015-17807) [n:127.0.0.1:52304_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52274/collection1/&wt=javabin&version=2}{add=[0 (1558937442130591744)]} 0 1
[junit4] 2> 2901507 INFO (qtp707322001-17842) [n:127.0.0.1:52309_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:52274/collection1/&wt=javabin&version=2}{add=[0 (1558937442130591744)]} 0 1
[junit4] 2> 2901507 INFO (qtp1646759755-17777) [n:127.0.0.1:52274_ c:collection1 s:shard1 r:core_nod
[...truncated too long message...]
kins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/response/BinaryResponseWriter.java (at line 51)
[ecj-lint] new JavaBinCodec(resolver).setWritableDocFields(resolver).marshal(response.getValues(), out);
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] 24. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/response/BinaryResponseWriter.java (at line 163)
[ecj-lint] new JavaBinCodec(resolver).setWritableDocFields(resolver).marshal(rsp.getValues(), out);
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] 25. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/response/BinaryResponseWriter.java (at line 166)
[ecj-lint] return (NamedList<Object>) new JavaBinCodec(resolver).unmarshal(in);
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 26. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/response/SmileResponseWriter.java (at line 33)
[ecj-lint] new SmileWriter(out, request, response).writeResponse();
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 27. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/schema/OpenExchangeRatesOrgProvider.java (at line 146)
[ecj-lint] ratesJsonStream = resourceLoader.openResource(ratesFileLocation);
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'ratesJsonStream' is not closed at this location
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 28. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/search/CollapsingQParserPlugin.java (at line 1277)
[ecj-lint] UninvertingReader uninvertingReader = new UninvertingReader(new ReaderWrapper(searcher.getSlowAtomicReader(), collapseField), mapping);
[ecj-lint] ^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'uninvertingReader' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 29. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/search/CursorMark.java (at line 188)
[ecj-lint] pieces = (List<Object>) new JavaBinCodec().unmarshal(in);
[ecj-lint] ^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] 30. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/search/CursorMark.java (at line 265)
[ecj-lint] new JavaBinCodec().marshal(marshalledValues, out);
[ecj-lint] ^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 31. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/search/DocSlice.java (at line 145)
[ecj-lint] HashDocSet h = new HashDocSet(docs,offset,len);
[ecj-lint] ^
[ecj-lint] Resource leak: 'h' is never closed
[ecj-lint] ----------
[ecj-lint] 32. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/search/DocSlice.java (at line 154)
[ecj-lint] HashDocSet h = new HashDocSet(docs,offset,len);
[ecj-lint] ^
[ecj-lint] Resource leak: 'h' is never closed
[ecj-lint] ----------
[ecj-lint] 33. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/search/DocSlice.java (at line 163)
[ecj-lint] HashDocSet h = new HashDocSet(docs,offset,len);
[ecj-lint] ^
[ecj-lint] Resource leak: 'h' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 34. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/search/function/OrdFieldSource.java (at line 78)
[ecj-lint] SolrIndexSearcher is = (SolrIndexSearcher) o;
[ecj-lint] ^^
[ecj-lint] Resource leak: 'is' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 35. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/search/function/ReverseOrdFieldSource.java (at line 78)
[ecj-lint] SolrIndexSearcher is = (SolrIndexSearcher) o;
[ecj-lint] ^^
[ecj-lint] Resource leak: 'is' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 36. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/servlet/SolrDispatchFilter.java (at line 263)
[ecj-lint] throw new SolrException(ErrorCode.SERVER_ERROR, "Error occurred while loading solr.xml from zookeeper", e);
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'loader' is not closed at this location
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 37. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/update/CdcrTransactionLog.java (at line 98)
[ecj-lint] ChannelFastInputStream is = new ChannelFastInputStream(channel, pos);
[ecj-lint] ^^
[ecj-lint] Resource leak: 'is' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 38. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/update/HdfsTransactionLog.java (at line 189)
[ecj-lint] LogCodec codec = new LogCodec(resolver);
[ecj-lint] ^^^^^
[ecj-lint] Resource leak: 'codec' is never closed
[ecj-lint] ----------
[ecj-lint] 39. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/update/HdfsTransactionLog.java (at line 259)
[ecj-lint] LogCodec codec = new LogCodec(resolver);
[ecj-lint] ^^^^^
[ecj-lint] Resource leak: 'codec' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 40. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 226)
[ecj-lint] ChannelFastInputStream is = new ChannelFastInputStream(channel, pos);
[ecj-lint] ^^
[ecj-lint] Resource leak: 'is' is never closed
[ecj-lint] ----------
[ecj-lint] 41. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 226)
[ecj-lint] ChannelFastInputStream is = new ChannelFastInputStream(channel, pos);
[ecj-lint] ^^
[ecj-lint] Resource leak: 'is' is never closed
[ecj-lint] ----------
[ecj-lint] 42. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 258)
[ecj-lint] LogCodec codec = new LogCodec(resolver);
[ecj-lint] ^^^^^
[ecj-lint] Resource leak: 'codec' is never closed
[ecj-lint] ----------
[ecj-lint] 43. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 273)
[ecj-lint] LogCodec codec = new LogCodec(resolver);
[ecj-lint] ^^^^^
[ecj-lint] Resource leak: 'codec' is never closed
[ecj-lint] ----------
[ecj-lint] 44. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 527)
[ecj-lint] LogCodec codec = new LogCodec(resolver);
[ecj-lint] ^^^^^
[ecj-lint] Resource leak: 'codec' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 45. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/util/SimplePostTool.java (at line 655)
[ecj-lint] BAOS bos = new BAOS();
[ecj-lint] ^^^
[ecj-lint] Resource leak: 'bos' is never closed
[ecj-lint] ----------
[ecj-lint] 46. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/util/SimplePostTool.java (at line 655)
[ecj-lint] BAOS bos = new BAOS();
[ecj-lint] ^^^
[ecj-lint] Resource leak: 'bos' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 47. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/util/SolrLogLayout.java (at line 149)
[ecj-lint] SolrQueryRequest req = requestInfo == null ? null : requestInfo.getReq();
[ecj-lint] ^^^
[ecj-lint] Resource leak: 'req' is never closed
[ecj-lint] ----------
[ecj-lint] 47 problems (2 errors, 45 warnings)
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/build.xml:775: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/build.xml:101: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build.xml:674: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/common-build.xml:2004: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/common-build.xml:2043: Compile failed; see the compiler error output for details.
Total time: 121 minutes 45 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3822
- Failure!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3822/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC
1 tests failed.
FAILED: org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication
Error Message:
[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-030/./collection1/data, /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-030/./collection1/data/index.20170209141348284, /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-030/./collection1/data/snapshot_metadata, /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-030/./collection1/data/index.20170209141348611] expected:<3> but was:<4>
Stack Trace:
java.lang.AssertionError: [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-030/./collection1/data, /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-030/./collection1/data/index.20170209141348284, /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-030/./collection1/data/snapshot_metadata, /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-030/./collection1/data/index.20170209141348611] expected:<3> but was:<4>
at __randomizedtesting.SeedInfo.seed([8B7EB6154EEE9495:7C0D584D88063B73]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.handler.TestReplicationHandler.checkForSingleIndex(TestReplicationHandler.java:918)
at org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication(TestReplicationHandler.java:1350)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11893 lines...]
[junit4] Suite: org.apache.solr.handler.TestReplicationHandler
[junit4] 2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/init-core-data-001
[junit4] 2> 2067239 INFO (SUITE-TestReplicationHandler-seed#[8B7EB6154EEE9495]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 2067246 INFO (SUITE-TestReplicationHandler-seed#[8B7EB6154EEE9495]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None) w/ MAC_OS_X supressed clientAuth
[junit4] 2> 2067247 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testEmptyCommits
[junit4] 2> 2067248 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-001/collection1
[junit4] 2> 2067302 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2067305 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6305aa0d{/solr,null,AVAILABLE}
[junit4] 2> 2067308 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5a6660d9{HTTP/1.1,[http/1.1]}{127.0.0.1:60953}
[junit4] 2> 2067308 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.e.j.s.Server Started @2072161ms
[junit4] 2> 2067308 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=60953}
[junit4] 2> 2067309 ERROR (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2067309 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 2067309 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 2067309 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2067310 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-09T21:12:12.251Z
[junit4] 2> 2067311 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-001/solr.xml
[junit4] 2> 2067465 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-001/.
[junit4] 2> 2067466 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2067518 INFO (coreLoadExecutor-5972-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2067575 INFO (coreLoadExecutor-5972-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2067597 INFO (coreLoadExecutor-5972-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 2067601 INFO (coreLoadExecutor-5972-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-001/./collection1
[junit4] 2> 2067603 INFO (coreLoadExecutor-5972-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-001/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-001/./collection1/data/]
[junit4] 2> 2067605 INFO (coreLoadExecutor-5972-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=975961123, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 2067744 INFO (coreLoadExecutor-5972-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2067744 INFO (coreLoadExecutor-5972-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2067745 INFO (coreLoadExecutor-5972-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=23, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 2067745 INFO (coreLoadExecutor-5972-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6a72970d[collection1] main]
[junit4] 2> 2067746 INFO (coreLoadExecutor-5972-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-001/collection1/conf
[junit4] 2> 2067747 INFO (coreLoadExecutor-5972-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
[junit4] 2> 2067747 INFO (coreLoadExecutor-5972-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2067748 INFO (searcherExecutor-5973-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6a72970d[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2068138 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-002/collection1
[junit4] 2> 2068148 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2068151 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1910af8b{/solr,null,AVAILABLE}
[junit4] 2> 2068151 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@28bcc483{HTTP/1.1,[http/1.1]}{127.0.0.1:60954}
[junit4] 2> 2068151 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.e.j.s.Server Started @2073004ms
[junit4] 2> 2068151 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-002/collection1/data, hostContext=/solr, hostPort=60954}
[junit4] 2> 2068152 ERROR (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2068152 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 2068152 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 2068152 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2068152 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-09T21:12:13.093Z
[junit4] 2> 2068153 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-002/solr.xml
[junit4] 2> 2068221 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-002/.
[junit4] 2> 2068221 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2068253 INFO (coreLoadExecutor-5982-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2068271 INFO (coreLoadExecutor-5982-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2068276 INFO (coreLoadExecutor-5982-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 2068281 INFO (coreLoadExecutor-5982-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-002/./collection1
[junit4] 2> 2068281 INFO (coreLoadExecutor-5982-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-002/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-002/./collection1/data/]
[junit4] 2> 2068284 INFO (coreLoadExecutor-5982-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=975961123, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 2068344 INFO (coreLoadExecutor-5982-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2068344 INFO (coreLoadExecutor-5982-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2068344 INFO (coreLoadExecutor-5982-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=23, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 2068344 INFO (coreLoadExecutor-5982-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5d1667e4[collection1] main]
[junit4] 2> 2068345 INFO (coreLoadExecutor-5982-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-002/collection1/conf
[junit4] 2> 2068349 INFO (coreLoadExecutor-5982-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
[junit4] 2> 2068349 INFO (searcherExecutor-5983-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5d1667e4[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2068718 INFO (qtp1189334030-13611) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/select params={q=*:*&sort=id+desc&wt=javabin&version=2} hits=0 status=0 QTime=1
[junit4] 2> 2068722 INFO (qtp1189334030-13612) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1]} 0 2
[junit4] 2> 2068723 INFO (qtp1189334030-13613) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2068723 INFO (qtp1189334030-13613) [ x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1d15f519
[junit4] 2> 2068730 INFO (qtp1189334030-13613) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1a1f489e[collection1] main]
[junit4] 2> 2068731 INFO (qtp1189334030-13613) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2068731 INFO (searcherExecutor-5973-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1a1f489e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C1)))}
[junit4] 2> 2068731 INFO (qtp1189334030-13613) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={commit=true&wt=javabin&version=2}{commit=} 0 8
[junit4] 2> 2068737 WARN (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher 'masterUrl' must be specified without the /replication suffix
[junit4] 2> 2068740 INFO (qtp1189334030-13615) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2068740 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Master's generation: 2
[junit4] 2> 2068740 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Master's version: 1486674733664
[junit4] 2> 2068740 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 2068740 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Starting replication process
[junit4] 2> 2068749 INFO (qtp1189334030-13616) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&wt=javabin&version=2&command=filelist} status=0 QTime=8
[junit4] 2> 2068750 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Number of files in latest index in master: 10
[junit4] 2> 2068754 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=40, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.719632895297728]
[junit4] 2> 2068755 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 2068756 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Starting download (fullCopy=false) to MetricsDirectory(MockDirectoryWrapper(RAMDirectory@2cba83ff lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7900a8da))
[junit4] 2> 2068759 INFO (qtp1189334030-13609) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.si&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2068764 INFO (qtp1189334030-13611) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_LuceneFixedGap_0.tib&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2068778 INFO (qtp1189334030-13612) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2068783 INFO (qtp1189334030-13613) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdx&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2068790 INFO (qtp1189334030-13614) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_LuceneFixedGap_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2068797 INFO (qtp1189334030-13615) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdt&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2068812 INFO (qtp1189334030-13616) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fnm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2068820 INFO (qtp1189334030-13609) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2068826 INFO (qtp1189334030-13611) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_LuceneFixedGap_0.tii&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2068833 INFO (qtp1189334030-13612) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=segments_2&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2068836 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Total time taken for download (fullCopy=false,bytesDownloaded=1687) : 0 secs (null bytes/sec) to MetricsDirectory(MockDirectoryWrapper(RAMDirectory@2cba83ff lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7900a8da))
[junit4] 2> 2068839 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=33, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=39.2919921875, floorSegmentMB=0.515625, forceMergeDeletesPctAllowed=1.0805423420591964, segmentsPerTier=24.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1937048169923884
[junit4] 2> 2068842 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 2068843 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6fbaa80b[collection1] main]
[junit4] 2> 2068843 INFO (searcherExecutor-5983-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6fbaa80b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C1)))}
[junit4] 2> 2068843 INFO (qtp1748937838-13628) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={wait=true&masterUrl=http://127.0.0.1:60953/solr/collection1/replication&command=fetchindex} status=0 QTime=108
[junit4] 2> 2068847 INFO (qtp1748937838-13630) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/select params={q=name:empty1&sort=id+desc&wt=javabin&version=2} hits=1 status=0 QTime=1
[junit4] 2> 2068848 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.h.TestReplicationHandler Waited for 0ms and found 1 docs
[junit4] 2> 2068849 INFO (qtp1189334030-13613) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={_trace=getDetails&qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2068849 INFO (qtp1189334030-13614) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={_trace=getDetails&qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2068853 INFO (qtp1189334030-13615) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={slave=false&qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2068854 INFO (qtp1748937838-13631) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={_trace=getDetails&qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=2
[junit4] 2> 2068854 INFO (qtp1189334030-13616) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={_trace=assertVersions&qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2068855 INFO (qtp1748937838-13632) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={_trace=assertVersions&qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2068855 INFO (qtp1189334030-13616) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2068856 INFO (qtp1189334030-13616) [ x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 2068856 INFO (qtp1189334030-13616) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2068856 INFO (qtp1189334030-13616) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={commit=true&wt=javabin&version=2}{commit=} 0 0
[junit4] 2> 2068857 WARN (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher 'masterUrl' must be specified without the /replication suffix
[junit4] 2> 2068859 INFO (qtp1189334030-13611) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2068859 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Master's generation: 2
[junit4] 2> 2068859 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Master's version: 1486674733664
[junit4] 2> 2068859 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Slave's generation: 2
[junit4] 2> 2068859 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Slave in sync with master.
[junit4] 2> 2068859 INFO (qtp1748937838-13632) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={wait=true&masterUrl=http://127.0.0.1:60953/solr/collection1/replication&command=fetchindex} status=0 QTime=2
[junit4] 2> 2068860 INFO (qtp1189334030-13612) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2]} 0 0
[junit4] 2> 2068861 INFO (qtp1189334030-13613) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2068861 INFO (qtp1189334030-13613) [ x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1d15f519
[junit4] 2> 2068864 INFO (qtp1189334030-13613) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@661a393a[collection1] main]
[junit4] 2> 2068864 INFO (qtp1189334030-13613) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2068864 INFO (searcherExecutor-5973-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@661a393a[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C1) Uninverting(_1(7.0.0):C1)))}
[junit4] 2> 2068864 INFO (qtp1189334030-13613) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={commit=true&wt=javabin&version=2}{commit=} 0 3
[junit4] 2> 2068866 WARN (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher 'masterUrl' must be specified without the /replication suffix
[junit4] 2> 2068867 INFO (qtp1189334030-13614) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2068867 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Master's generation: 3
[junit4] 2> 2068867 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Master's version: 1486674733802
[junit4] 2> 2068867 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Slave's generation: 2
[junit4] 2> 2068867 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Starting replication process
[junit4] 2> 2068869 INFO (qtp1189334030-13615) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=3&qt=/replication&wt=javabin&version=2&command=filelist} status=0 QTime=1
[junit4] 2> 2068869 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Number of files in latest index in master: 19
[junit4] 2> 2068871 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=40, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.719632895297728]
[junit4] 2> 2068872 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 2068873 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Starting download (fullCopy=false) to MetricsDirectory(MockDirectoryWrapper(RAMDirectory@3efc638b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1ec7e612))
[junit4] 2> 2068873 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _0.si because it already exists
[junit4] 2> 2068873 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _0_LuceneFixedGap_0.tib because it already exists
[junit4] 2> 2068873 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _0.nvd because it already exists
[junit4] 2> 2068873 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _0.fdx because it already exists
[junit4] 2> 2068873 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _0_LuceneFixedGap_0.doc because it already exists
[junit4] 2> 2068874 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _0.fdt because it already exists
[junit4] 2> 2068874 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _0.fnm because it already exists
[junit4] 2> 2068874 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _0.nvm because it already exists
[junit4] 2> 2068874 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _0_LuceneFixedGap_0.tii because it already exists
[junit4] 2> 2068876 INFO (qtp1189334030-13609) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=3&qt=/replication&file=_1_LuceneFixedGap_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2068880 INFO (qtp1189334030-13616) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=3&qt=/replication&file=_1.si&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2068881 INFO (qtp1189334030-13611) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=3&qt=/replication&file=_1_LuceneFixedGap_0.tii&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2068882 INFO (qtp1189334030-13612) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=3&qt=/replication&file=_1.nvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2068884 INFO (qtp1189334030-13613) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=3&qt=/replication&file=_1.fdx&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2068885 INFO (qtp1189334030-13614) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=3&qt=/replication&file=_1.fnm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2068886 INFO (qtp1189334030-13615) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=3&qt=/replication&file=_1.nvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2068887 INFO (qtp1189334030-13609) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=3&qt=/replication&file=_1_LuceneFixedGap_0.tib&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2068888 INFO (qtp1189334030-13616) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=3&qt=/replication&file=_1.fdt&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2068889 INFO (qtp1189334030-13611) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=3&qt=/replication&file=segments_3&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2068890 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Total time taken for download (fullCopy=false,bytesDownloaded=1750) : 0 secs (null bytes/sec) to MetricsDirectory(MockDirectoryWrapper(RAMDirectory@3efc638b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1ec7e612))
[junit4] 2> 2068892 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=33, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=39.2919921875, floorSegmentMB=0.515625, forceMergeDeletesPctAllowed=1.0805423420591964, segmentsPerTier=24.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1937048169923884
[junit4] 2> 2068894 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 2068895 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7b4209bf[collection1] main]
[junit4] 2> 2068896 INFO (searcherExecutor-5983-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7b4209bf[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C1) Uninverting(_1(7.0.0):C1)))}
[junit4] 2> 2068896 INFO (qtp1748937838-13626) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={wait=true&masterUrl=http://127.0.0.1:60953/solr/collection1/replication&command=fetchindex} status=0 QTime=30
[junit4] 2> 2068898 INFO (qtp1748937838-13628) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/select params={q=name:empty2&sort=id+desc&wt=javabin&version=2} hits=1 status=0 QTime=0
[junit4] 2> 2068898 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.h.TestReplicationHandler Waited for 0ms and found 1 docs
[junit4] 2> 2068899 INFO (qtp1189334030-13612) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={_trace=getDetails&qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2068900 INFO (qtp1189334030-13613) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={_trace=getDetails&qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2068901 INFO (qtp1189334030-13614) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={slave=false&qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2068902 INFO (qtp1748937838-13629) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={_trace=getDetails&qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
[junit4] 2> 2068902 INFO (qtp1189334030-13615) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={_trace=assertVersions&qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2068903 INFO (qtp1748937838-13630) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={_trace=assertVersions&qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2068904 INFO (qtp1189334030-13609) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[3]} 0 1
[junit4] 2> 2068905 INFO (qtp1189334030-13616) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2068905 INFO (qtp1189334030-13616) [ x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1d15f519
[junit4] 2> 2068909 INFO (qtp1189334030-13616) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2ed4abc1[collection1] realtime]
[junit4] 2> 2068909 INFO (qtp1189334030-13616) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2068909 INFO (qtp1189334030-13616) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={openSearcher=false&commit=true&wt=javabin&version=2}{commit=} 0 3
[junit4] 2> 2068911 WARN (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher 'masterUrl' must be specified without the /replication suffix
[junit4] 2> 2068912 INFO (qtp1189334030-13611) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2068912 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Master's generation: 4
[junit4] 2> 2068912 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Master's version: 1486674733846
[junit4] 2> 2068912 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Slave's generation: 3
[junit4] 2> 2068912 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Starting replication process
[junit4] 2> 2068914 INFO (qtp1189334030-13612) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=4&qt=/replication&wt=javabin&version=2&command=filelist} status=0 QTime=1
[junit4] 2> 2068914 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Number of files in latest index in master: 28
[junit4] 2> 2068916 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=40, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.719632895297728]
[junit4] 2> 2068917 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 2068918 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Found unused file: segments_2
[junit4] 2> 2068918 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Sleeping for 1000ms to wait for unused lucene index files to be delete-able
[junit4] 2> 2069929 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Found unused file: segments_2
[junit4] 2> 2069929 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Sleeping for 1000ms to wait for unused lucene index files to be delete-able
[junit4] 2> 2070940 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Found unused file: segments_2
[junit4] 2> 2070940 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Sleeping for 1000ms to wait for unused lucene index files to be delete-able
[junit4] 2> 2071952 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Found unused file: segments_2
[junit4] 2> 2071952 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Sleeping for 1000ms to wait for unused lucene index files to be delete-able
[junit4] 2> 2072983 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Found unused file: segments_2
[junit4] 2> 2072983 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Sleeping for 1000ms to wait for unused lucene index files to be delete-able
[junit4] 2> 2074000 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Found unused file: segments_2
[junit4] 2> 2074000 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Sleeping for 1000ms to wait for unused lucene index files to be delete-able
[junit4] 2> 2075011 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Found unused file: segments_2
[junit4] 2> 2075011 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Sleeping for 1000ms to wait for unused lucene index files to be delete-able
[junit4] 2> 2076028 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Found unused file: segments_2
[junit4] 2> 2076028 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Sleeping for 1000ms to wait for unused lucene index files to be delete-able
[junit4] 2> 2077049 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Found unused file: segments_2
[junit4] 2> 2077049 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Sleeping for 1000ms to wait for unused lucene index files to be delete-able
[junit4] 2> 2078054 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Found unused file: segments_2
[junit4] 2> 2078054 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Sleeping for 1000ms to wait for unused lucene index files to be delete-able
[junit4] 2> 2079065 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Found unused file: segments_2
[junit4] 2> 2079065 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Sleeping for 1000ms to wait for unused lucene index files to be delete-able
[junit4] 2> 2080066 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher IndexFetcher slept for 11000ms for unused lucene index files to be delete-able
[junit4] 2> 2080066 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Starting download (fullCopy=false) to MetricsDirectory(MockDirectoryWrapper(RAMDirectory@649fdbe lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@28f7b805))
[junit4] 2> 2080067 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _0.si because it already exists
[junit4] 2> 2080067 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _0_LuceneFixedGap_0.tib because it already exists
[junit4] 2> 2080067 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _0.nvd because it already exists
[junit4] 2> 2080067 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _0.fdx because it already exists
[junit4] 2> 2080068 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _0_LuceneFixedGap_0.doc because it already exists
[junit4] 2> 2080069 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _0.fdt because it already exists
[junit4] 2> 2080069 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _0.fnm because it already exists
[junit4] 2> 2080069 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _0.nvm because it already exists
[junit4] 2> 2080069 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _0_LuceneFixedGap_0.tii because it already exists
[junit4] 2> 2080069 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _1_LuceneFixedGap_0.doc because it already exists
[junit4] 2> 2080069 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _1.si because it already exists
[junit4] 2> 2080069 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _1_LuceneFixedGap_0.tii because it already exists
[junit4] 2> 2080069 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _1.nvd because it already exists
[junit4] 2> 2080070 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _1.fdx because it already exists
[junit4] 2> 2080070 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _1.fnm because it already exists
[junit4] 2> 2080070 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _1.nvm because it already exists
[junit4] 2> 2080070 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _1_LuceneFixedGap_0.tib because it already exists
[junit4] 2> 2080070 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _1.fdt because it already exists
[junit4] 2> 2080090 INFO (qtp1189334030-13613) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=4&qt=/replication&file=_2.nvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2080094 INFO (qtp1189334030-13614) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=4&qt=/replication&file=_2_LuceneFixedGap_0.tib&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2080096 INFO (qtp1189334030-13615) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=4&qt=/replication&file=_2.si&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2080098 INFO (qtp1189334030-13609) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=4&qt=/replication&file=_2_LuceneFixedGap_0.tii&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2080100 INFO (qtp1189334030-13616) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=4&qt=/replication&file=_2.fdt&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2080109 INFO (qtp1189334030-13611) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=4&qt=/replication&file=_2.fnm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2080110 INFO (qtp1189334030-13612) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=4&qt=/replication&file=_2.nvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2080111 INFO (qtp1189334030-13613) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=4&qt=/replication&file=_2_LuceneFixedGap_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2080112 INFO (qtp1189334030-13614) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=4&qt=/replication&file=_2.fdx&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2080114 INFO (qtp1189334030-13615) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=4&qt=/replication&file=segments_4&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2080115 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Total time taken for download (fullCopy=false,bytesDownloaded=1813) : 11 secs (164 bytes/sec) to MetricsDirectory(MockDirectoryWrapper(RAMDirectory@649fdbe lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@28f7b805))
[junit4] 2> 2080124 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=33, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=39.2919921875, floorSegmentMB=0.515625, forceMergeDeletesPctAllowed=1.0805423420591964, segmentsPerTier=24.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1937048169923884
[junit4] 2> 2080127 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 2080130 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@78a19d62[collection1] main]
[junit4] 2> 2080130 INFO (searcherExecutor-5983-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@78a19d62[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C1) Uninverting(_1(7.0.0):C1) Uninverting(_2(7.0.0):C1)))}
[junit4] 2> 2080131 INFO (qtp1748937838-13631) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={wait=true&masterUrl=http://127.0.0.1:60953/solr/collection1/replication&command=fetchindex} status=0 QTime=11220
[junit4] 2> 2080135 INFO (qtp1189334030-13609) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/select params={q=name:empty3&sort=id+desc&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 2080135 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.h.TestReplicationHandler Waited for 0ms and found 0 docs
[junit4] 2> 2080138 INFO (qtp1748937838-13633) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/select params={q=name:empty3&sort=id+desc&wt=javabin&version=2} hits=1 status=0 QTime=0
[junit4] 2> 2080138 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.h.TestReplicationHandler Waited for 0ms and found 1 docs
[junit4] 2> 2080139 INFO (qtp1189334030-13616) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[4]} 0 0
[junit4] 2> 2080140 INFO (qtp1189334030-13611) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2080140 INFO (qtp1189334030-13611) [ x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1d15f519
[junit4] 2> 2080156 INFO (qtp1189334030-13611) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@53d8b289[collection1] main]
[junit4] 2> 2080156 INFO (qtp1189334030-13611) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2080156 INFO (searcherExecutor-5973-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@53d8b289[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C1) Uninverting(_1(7.0.0):C1) Uninverting(_2(7.0.0):C1) Uninverting(_3(7.0.0):C1)))}
[junit4] 2> 2080156 INFO (qtp1189334030-13611) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={commit=true&wt=javabin&version=2}{commit=} 0 16
[junit4] 2> 2080158 WARN (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher 'masterUrl' must be specified without the /replication suffix
[junit4] 2> 2080159 INFO (qtp1189334030-13612) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2080159 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Master's generation: 5
[junit4] 2> 2080159 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Master's version: 1486674745081
[junit4] 2> 2080159 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Slave's generation: 4
[junit4] 2> 2080159 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Starting replication process
[junit4] 2> 2080187 INFO (qtp1189334030-13613) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=5&qt=/replication&wt=javabin&version=2&command=filelist} status=0 QTime=27
[junit4] 2> 2080187 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Number of files in latest index in master: 37
[junit4] 2> 2080258 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=40, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.719632895297728]
[junit4] 2> 2080266 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 2080278 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Starting download (fullCopy=false) to MetricsDirectory(MockDirectoryWrapper(RAMDirectory@72e38522 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4982cf0a))
[junit4] 2> 2080288 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _0.si because it already exists
[junit4] 2> 2080288 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _0_LuceneFixedGap_0.tib because it already exists
[junit4] 2> 2080298 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _0.nvd because it already exists
[junit4] 2> 2080309 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _0.fdx because it already exists
[junit4] 2> 2080319 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _0_LuceneFixedGap_0.doc because it already exists
[junit4] 2> 2080319 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _0.fdt because it already exists
[junit4] 2> 2080331 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _0.fnm because it already exists
[junit4] 2> 2080332 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _0.nvm because it already exists
[junit4] 2> 2080334 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _0_LuceneFixedGap_0.tii because it already exists
[junit4] 2> 2080334 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _1_LuceneFixedGap_0.doc because it already exists
[junit4] 2> 2080335 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _1.si because it already exists
[junit4] 2> 2080335 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _1_LuceneFixedGap_0.tii because it already exists
[junit4] 2> 2080336 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _1.nvd because it already exists
[junit4] 2> 2080343 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _1.fdx because it already exists
[junit4] 2> 2080344 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _1.fnm because it already exists
[junit4] 2> 2080349 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _1.nvm because it already exists
[junit4] 2> 2080350 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _1_LuceneFixedGap_0.tib because it already exists
[junit4] 2> 2080350 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _1.fdt because it already exists
[junit4] 2> 2080350 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _2.nvd because it already exists
[junit4] 2> 2080353 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _2_LuceneFixedGap_0.tib because it already exists
[junit4] 2> 2080353 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _2.si because it already exists
[junit4] 2> 2080353 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _2_LuceneFixedGap_0.tii because it already exists
[junit4] 2> 2080353 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _2.fdt because it already exists
[junit4] 2> 2080354 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _2.fnm because it already exists
[junit4] 2> 2080354 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _2.nvm because it already exists
[junit4] 2> 2080354 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _2_LuceneFixedGap_0.doc because it already exists
[junit4] 2> 2080354 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Skipping download for _2.fdx because it already exists
[junit4] 2> 2080361 INFO (qtp1189334030-13614) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=5&qt=/replication&file=_3.fdx&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2080365 INFO (qtp1189334030-13615) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=5&qt=/replication&file=_3_LuceneFixedGap_0.tii&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2080368 INFO (qtp1189334030-13609) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=5&qt=/replication&file=_3.si&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2080372 INFO (qtp1189334030-13616) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=5&qt=/replication&file=_3.nvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2080374 INFO (qtp1189334030-13611) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=5&qt=/replication&file=_3_LuceneFixedGap_0.tib&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2080376 INFO (qtp1189334030-13612) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=5&qt=/replication&file=_3.fnm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2080378 INFO (qtp1189334030-13613) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=5&qt=/replication&file=_3.nvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2080379 INFO (qtp1189334030-13614) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=5&qt=/replication&file=_3.fdt&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2080391 INFO (qtp1189334030-13615) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=5&qt=/replication&file=_3_LuceneFixedGap_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2080393 INFO (qtp1189334030-13609) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=5&qt=/replication&file=segments_5&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2080393 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Total time taken for download (fullCopy=false,bytesDownloaded=1876) : 0 secs (null bytes/sec) to MetricsDirectory(MockDirectoryWrapper(RAMDirectory@72e38522 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4982cf0a))
[junit4] 2> 2080414 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=33, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=39.2919921875, floorSegmentMB=0.515625, forceMergeDeletesPctAllowed=1.0805423420591964, segmentsPerTier=24.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1937048169923884
[junit4] 2> 2080423 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 2080453 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2d92b7e5[collection1] main]
[junit4] 2> 2080453 INFO (searcherExecutor-5983-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2d92b7e5[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C1) Uninverting(_1(7.0.0):C1) Uninverting(_2(7.0.0):C1) Uninverting(_3(7.0.0):C1)))}
[junit4] 2> 2080454 INFO (qtp1748937838-13632) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={wait=true&masterUrl=http://127.0.0.1:60953/solr/collection1/replication&command=fetchindex} status=0 QTime=296
[junit4] 2> 2080458 INFO (qtp1189334030-13616) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/select params={q=name:(empty1+empty2+empty3+empty4)&sort=id+desc&wt=javabin&version=2} hits=4 status=0 QTime=2
[junit4] 2> 2080459 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.h.TestReplicationHandler Waited for 0ms and found 4 docs
[junit4] 2> 2080460 INFO (qtp1748937838-13626) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/select params={q=name:(empty1+empty2+empty3+empty4)&sort=id+desc&wt=javabin&version=2} hits=4 status=0 QTime=1
[junit4] 2> 2080461 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.h.TestReplicationHandler Waited for 0ms and found 4 docs
[junit4] 2> 2080519 INFO (qtp1189334030-13611) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={_trace=getDetails&qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=57
[junit4] 2> 2080522 INFO (qtp1189334030-13612) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={_trace=getDetails&qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=2
[junit4] 2> 2080528 INFO (qtp1189334030-13613) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={slave=false&qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=3
[junit4] 2> 2080528 INFO (qtp1748937838-13628) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={_trace=getDetails&qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=5
[junit4] 2> 2080529 INFO (qtp1189334030-13614) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={_trace=assertVersions&qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2080530 INFO (qtp1748937838-13629) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={_trace=assertVersions&qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2080530 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testEmptyCommits
[junit4] 2> 2080531 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@5a6660d9{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2080531 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=376123576
[junit4] 2> 2080532 INFO (coreCloseExecutor-5992-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@7b12171a
[junit4] 2> 2080533 INFO (coreCloseExecutor-5992-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection1
[junit4] 2> 2080534 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 2080541 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6305aa0d{/solr,null,UNAVAILABLE}
[junit4] 2> 2080543 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@28bcc483{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2080543 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1516804334
[junit4] 2> 2080544 INFO (coreCloseExecutor-5994-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@44813ceb
[junit4] 2> 2080545 INFO (coreCloseExecutor-5994-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection1
[junit4] 2> 2080546 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 2080547 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[8B7EB6154EEE9495]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1910af8b{/solr,null,UNAVAILABLE}
[junit4] 2> 2080551 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.a.s.SolrTestCaseJ4 ###Starting doTestIndexFetchWithMasterUrl
[junit4] 2> 2080552 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-003/collection1
[junit4] 2> 2080567 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2080569 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@53aa1c70{/solr,null,AVAILABLE}
[junit4] 2> 2080569 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@684e84fd{HTTP/1.1,[http/1.1]}{127.0.0.1:61010}
[junit4] 2> 2080569 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.e.j.s.Server Started @2085422ms
[junit4] 2> 2080570 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-003/collection1/data, hostContext=/solr, hostPort=61010}
[junit4] 2> 2080570 ERROR (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2080570 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 2080570 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 2080570 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2080570 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-09T21:12:25.511Z
[junit4] 2> 2080572 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-003/solr.xml
[junit4] 2> 2081137 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-003/.
[junit4] 2> 2081137 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2081156 INFO (coreLoadExecutor-6001-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2081163 INFO (coreLoadExecutor-6001-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2081166 INFO (coreLoadExecutor-6001-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 2081170 INFO (coreLoadExecutor-6001-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-003/./collection1
[junit4] 2> 2081170 INFO (coreLoadExecutor-6001-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-003/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-003/./collection1/data/]
[junit4] 2> 2081171 INFO (coreLoadExecutor-6001-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=975961123, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 2081229 INFO (coreLoadExecutor-6001-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2081229 INFO (coreLoadExecutor-6001-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2081230 INFO (coreLoadExecutor-6001-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=23, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 2081230 INFO (coreLoadExecutor-6001-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@bf36c29[collection1] main]
[junit4] 2> 2081230 INFO (coreLoadExecutor-6001-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-003/collection1/conf
[junit4] 2> 2081231 INFO (coreLoadExecutor-6001-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
[junit4] 2> 2081231 INFO (coreLoadExecutor-6001-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2081231 INFO (searcherExecutor-6002-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@bf36c29[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2081599 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-004/collection1
[junit4] 2> 2081618 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2081620 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@68d90b2d{/solr,null,AVAILABLE}
[junit4] 2> 2081621 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@9e09ab4{HTTP/1.1,[http/1.1]}{127.0.0.1:61011}
[junit4] 2> 2081621 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.e.j.s.Server Started @2086474ms
[junit4] 2> 2081621 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-004/collection1/data, hostContext=/solr, hostPort=61011}
[junit4] 2> 2081622 ERROR (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2081622 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 2081622 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 2081622 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2081622 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-09T21:12:26.563Z
[junit4] 2> 2081623 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-004/solr.xml
[junit4] 2> 2081682 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-004/.
[junit4] 2> 2081682 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2081697 INFO (coreLoadExecutor-6011-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2081710 INFO (coreLoadExecutor-6011-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2081718 INFO (coreLoadExecutor-6011-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 2081724 INFO (coreLoadExecutor-6011-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-004/./collection1
[junit4] 2> 2081725 INFO (coreLoadExecutor-6011-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-004/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-004/./collection1/data/]
[junit4] 2> 2081727 INFO (coreLoadExecutor-6011-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=975961123, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 2082016 INFO (coreLoadExecutor-6011-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2082016 INFO (coreLoadExecutor-6011-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2082016 INFO (coreLoadExecutor-6011-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=23, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 2082017 INFO (coreLoadExecutor-6011-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@45be4499[collection1] main]
[junit4] 2> 2082017 INFO (coreLoadExecutor-6011-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8B7EB6154EEE9495-001/solr-instance-004/collection1/conf
[junit4] 2> 2082020 INFO (coreLoadExecutor-6011-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
[junit4] 2> 2082020 INFO (searcherExecutor-6012-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@45be4499[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2082023 INFO (qtp1188327178-13662) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2082024 INFO (indexFetcher-6016-thread-1) [ x:collection1] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 2082024 INFO (indexFetcher-6016-thread-1) [ x:collection1] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 2082024 INFO (indexFetcher-6016-thread-1) [ x:collection1] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 2082533 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@9e09ab4{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2082533 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=357007182
[junit4] 2> 2082533 INFO (coreCloseExecutor-6017-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@53d72bc0
[junit4] 2> 2082535 INFO (coreCloseExecutor-6017-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection1
[junit4] 2> 2082536 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 2082537 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@68d90b2d{/solr,null,UNAVAILABLE}
[junit4] 2> 2082539 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2082540 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7d595bf4{/solr,null,AVAILABLE}
[junit4] 2> 2082541 INFO (TEST-TestReplicationHandler.doTestIndexFetchWithMasterUrl-seed#[8B7EB6154EEE9495]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@79234450{HTTP/1.1,[http/1.1]}{127.0.0.1:61013}
[junit4] 2> 2082541 INFO (TEST-TestReplicationHandler.doTestIndexFetchW
[...truncated too long message...]
kins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/response/BinaryResponseWriter.java (at line 51)
[ecj-lint] new JavaBinCodec(resolver).setWritableDocFields(resolver).marshal(response.getValues(), out);
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] 24. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/response/BinaryResponseWriter.java (at line 163)
[ecj-lint] new JavaBinCodec(resolver).setWritableDocFields(resolver).marshal(rsp.getValues(), out);
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] 25. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/response/BinaryResponseWriter.java (at line 166)
[ecj-lint] return (NamedList<Object>) new JavaBinCodec(resolver).unmarshal(in);
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 26. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/response/SmileResponseWriter.java (at line 33)
[ecj-lint] new SmileWriter(out, request, response).writeResponse();
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 27. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/schema/OpenExchangeRatesOrgProvider.java (at line 146)
[ecj-lint] ratesJsonStream = resourceLoader.openResource(ratesFileLocation);
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'ratesJsonStream' is not closed at this location
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 28. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/search/CollapsingQParserPlugin.java (at line 1277)
[ecj-lint] UninvertingReader uninvertingReader = new UninvertingReader(new ReaderWrapper(searcher.getSlowAtomicReader(), collapseField), mapping);
[ecj-lint] ^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'uninvertingReader' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 29. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/search/CursorMark.java (at line 188)
[ecj-lint] pieces = (List<Object>) new JavaBinCodec().unmarshal(in);
[ecj-lint] ^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] 30. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/search/CursorMark.java (at line 265)
[ecj-lint] new JavaBinCodec().marshal(marshalledValues, out);
[ecj-lint] ^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 31. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/search/DocSlice.java (at line 145)
[ecj-lint] HashDocSet h = new HashDocSet(docs,offset,len);
[ecj-lint] ^
[ecj-lint] Resource leak: 'h' is never closed
[ecj-lint] ----------
[ecj-lint] 32. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/search/DocSlice.java (at line 154)
[ecj-lint] HashDocSet h = new HashDocSet(docs,offset,len);
[ecj-lint] ^
[ecj-lint] Resource leak: 'h' is never closed
[ecj-lint] ----------
[ecj-lint] 33. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/search/DocSlice.java (at line 163)
[ecj-lint] HashDocSet h = new HashDocSet(docs,offset,len);
[ecj-lint] ^
[ecj-lint] Resource leak: 'h' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 34. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/search/function/OrdFieldSource.java (at line 78)
[ecj-lint] SolrIndexSearcher is = (SolrIndexSearcher) o;
[ecj-lint] ^^
[ecj-lint] Resource leak: 'is' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 35. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/search/function/ReverseOrdFieldSource.java (at line 78)
[ecj-lint] SolrIndexSearcher is = (SolrIndexSearcher) o;
[ecj-lint] ^^
[ecj-lint] Resource leak: 'is' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 36. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/servlet/SolrDispatchFilter.java (at line 263)
[ecj-lint] throw new SolrException(ErrorCode.SERVER_ERROR, "Error occurred while loading solr.xml from zookeeper", e);
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'loader' is not closed at this location
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 37. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/update/CdcrTransactionLog.java (at line 98)
[ecj-lint] ChannelFastInputStream is = new ChannelFastInputStream(channel, pos);
[ecj-lint] ^^
[ecj-lint] Resource leak: 'is' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 38. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/update/HdfsTransactionLog.java (at line 189)
[ecj-lint] LogCodec codec = new LogCodec(resolver);
[ecj-lint] ^^^^^
[ecj-lint] Resource leak: 'codec' is never closed
[ecj-lint] ----------
[ecj-lint] 39. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/update/HdfsTransactionLog.java (at line 259)
[ecj-lint] LogCodec codec = new LogCodec(resolver);
[ecj-lint] ^^^^^
[ecj-lint] Resource leak: 'codec' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 40. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 226)
[ecj-lint] ChannelFastInputStream is = new ChannelFastInputStream(channel, pos);
[ecj-lint] ^^
[ecj-lint] Resource leak: 'is' is never closed
[ecj-lint] ----------
[ecj-lint] 41. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 226)
[ecj-lint] ChannelFastInputStream is = new ChannelFastInputStream(channel, pos);
[ecj-lint] ^^
[ecj-lint] Resource leak: 'is' is never closed
[ecj-lint] ----------
[ecj-lint] 42. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 258)
[ecj-lint] LogCodec codec = new LogCodec(resolver);
[ecj-lint] ^^^^^
[ecj-lint] Resource leak: 'codec' is never closed
[ecj-lint] ----------
[ecj-lint] 43. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 273)
[ecj-lint] LogCodec codec = new LogCodec(resolver);
[ecj-lint] ^^^^^
[ecj-lint] Resource leak: 'codec' is never closed
[ecj-lint] ----------
[ecj-lint] 44. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 527)
[ecj-lint] LogCodec codec = new LogCodec(resolver);
[ecj-lint] ^^^^^
[ecj-lint] Resource leak: 'codec' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 45. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/util/SimplePostTool.java (at line 655)
[ecj-lint] BAOS bos = new BAOS();
[ecj-lint] ^^^
[ecj-lint] Resource leak: 'bos' is never closed
[ecj-lint] ----------
[ecj-lint] 46. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/util/SimplePostTool.java (at line 655)
[ecj-lint] BAOS bos = new BAOS();
[ecj-lint] ^^^
[ecj-lint] Resource leak: 'bos' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 47. WARNING in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/java/org/apache/solr/util/SolrLogLayout.java (at line 149)
[ecj-lint] SolrQueryRequest req = requestInfo == null ? null : requestInfo.getReq();
[ecj-lint] ^^^
[ecj-lint] Resource leak: 'req' is never closed
[ecj-lint] ----------
[ecj-lint] 47 problems (2 errors, 45 warnings)
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/build.xml:775: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/build.xml:101: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build.xml:674: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/common-build.xml:2004: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/common-build.xml:2043: Compile failed; see the compiler error output for details.
Total time: 117 minutes 18 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3821
- Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3821/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC
1 tests failed.
FAILED: org.apache.solr.handler.TestReplicationHandler.doTestDetails
Error Message:
Stack Trace:
java.lang.NullPointerException
at __randomizedtesting.SeedInfo.seed([EF0F17C9FF4091C6:955234BC6918B84E]:0)
at org.apache.solr.handler.TestReplicationHandler.doTestDetails(TestReplicationHandler.java:312)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11840 lines...]
[junit4] Suite: org.apache.solr.handler.TestReplicationHandler
[junit4] 2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_EF0F17C9FF4091C6-001/init-core-data-001
[junit4] 2> 2053112 INFO (SUITE-TestReplicationHandler-seed#[EF0F17C9FF4091C6]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 2053116 INFO (SUITE-TestReplicationHandler-seed#[EF0F17C9FF4091C6]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None) w/ MAC_OS_X supressed clientAuth
[junit4] 2> 2053118 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.SolrTestCaseJ4 ###Starting doTestReplicateAfterStartup
[junit4] 2> 2053119 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_EF0F17C9FF4091C6-001/solr-instance-001/collection1
[junit4] 2> 2053173 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2053176 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4e5de820{/solr,null,AVAILABLE}
[junit4] 2> 2053177 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6f0ff173{HTTP/1.1,[http/1.1]}{127.0.0.1:54335}
[junit4] 2> 2053178 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.e.j.s.Server Started @2058313ms
[junit4] 2> 2053178 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_EF0F17C9FF4091C6-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=54335}
[junit4] 2> 2053179 ERROR (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2053180 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 2053180 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 2053180 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2053180 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-09T08:12:49.900Z
[junit4] 2> 2053181 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_EF0F17C9FF4091C6-001/solr-instance-001/solr.xml
[junit4] 2> 2053739 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_EF0F17C9FF4091C6-001/solr-instance-001/.
[junit4] 2> 2053739 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2053769 WARN (coreLoadExecutor-5561-thread-1) [ x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 2053770 INFO (coreLoadExecutor-5561-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2053785 INFO (coreLoadExecutor-5561-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2053790 INFO (coreLoadExecutor-5561-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 2053796 INFO (coreLoadExecutor-5561-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_EF0F17C9FF4091C6-001/solr-instance-001/./collection1
[junit4] 2> 2053796 INFO (coreLoadExecutor-5561-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_EF0F17C9FF4091C6-001/solr-instance-001/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_EF0F17C9FF4091C6-001/solr-instance-001/./collection1/data/]
[junit4] 2> 2053797 INFO (coreLoadExecutor-5561-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=44, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.18221903121906655]
[junit4] 2> 2053937 INFO (coreLoadExecutor-5561-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2053937 INFO (coreLoadExecutor-5561-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2053937 INFO (coreLoadExecutor-5561-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=43, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=99.1337890625, floorSegmentMB=0.34375, forceMergeDeletesPctAllowed=19.395645586126445, segmentsPerTier=20.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 2053938 INFO (coreLoadExecutor-5561-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6e57090a[collection1] main]
[junit4] 2> 2053939 INFO (coreLoadExecutor-5561-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_EF0F17C9FF4091C6-001/solr-instance-001/collection1/conf
[junit4] 2> 2053940 INFO (coreLoadExecutor-5561-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
[junit4] 2> 2053940 INFO (coreLoadExecutor-5561-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2053940 INFO (searcherExecutor-5562-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6e57090a[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2055138 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_EF0F17C9FF4091C6-001/solr-instance-002/collection1
[junit4] 2> 2055163 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2055165 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@33138022{/solr,null,AVAILABLE}
[junit4] 2> 2055166 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@49c80262{HTTP/1.1,[http/1.1]}{127.0.0.1:54337}
[junit4] 2> 2055166 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.e.j.s.Server Started @2060301ms
[junit4] 2> 2055166 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_EF0F17C9FF4091C6-001/solr-instance-002/collection1/data, hostContext=/solr, hostPort=54337}
[junit4] 2> 2055167 ERROR (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2055167 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 2055167 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 2055168 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2055168 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-09T08:12:51.888Z
[junit4] 2> 2055182 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_EF0F17C9FF4091C6-001/solr-instance-002/solr.xml
[junit4] 2> 2055314 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_EF0F17C9FF4091C6-001/solr-instance-002/.
[junit4] 2> 2055314 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2055359 WARN (coreLoadExecutor-5571-thread-1) [ x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 2055360 INFO (coreLoadExecutor-5571-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2055370 INFO (coreLoadExecutor-5571-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2055376 INFO (coreLoadExecutor-5571-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 2055383 INFO (coreLoadExecutor-5571-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_EF0F17C9FF4091C6-001/solr-instance-002/./collection1
[junit4] 2> 2055383 INFO (coreLoadExecutor-5571-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_EF0F17C9FF4091C6-001/solr-instance-002/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_EF0F17C9FF4091C6-001/solr-instance-002/./collection1/data/]
[junit4] 2> 2055385 INFO (coreLoadExecutor-5571-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=44, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.18221903121906655]
[junit4] 2> 2055492 INFO (coreLoadExecutor-5571-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2055492 INFO (coreLoadExecutor-5571-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2055492 INFO (coreLoadExecutor-5571-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=43, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=99.1337890625, floorSegmentMB=0.34375, forceMergeDeletesPctAllowed=19.395645586126445, segmentsPerTier=20.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 2055505 INFO (coreLoadExecutor-5571-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2d6f8686[collection1] main]
[junit4] 2> 2055506 INFO (coreLoadExecutor-5571-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_EF0F17C9FF4091C6-001/solr-instance-002/collection1/conf
[junit4] 2> 2055513 INFO (coreLoadExecutor-5571-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
[junit4] 2> 2055514 INFO (searcherExecutor-5572-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2d6f8686[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2055751 INFO (qtp1875896903-11534) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2055751 INFO (indexFetcher-5576-thread-1) [ x:collection1] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 2055751 INFO (indexFetcher-5576-thread-1) [ x:collection1] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 2055751 INFO (indexFetcher-5576-thread-1) [ x:collection1] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 2056737 INFO (qtp1875896903-11535) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2056737 INFO (indexFetcher-5576-thread-1) [ x:collection1] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 2056737 INFO (indexFetcher-5576-thread-1) [ x:collection1] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 2056737 INFO (indexFetcher-5576-thread-1) [ x:collection1] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 2056789 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@49c80262{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2056789 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1117504763
[junit4] 2> 2056790 INFO (coreCloseExecutor-5577-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@42ece1f7
[junit4] 2> 2056791 INFO (coreCloseExecutor-5577-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection1
[junit4] 2> 2056792 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 2056792 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@33138022{/solr,null,UNAVAILABLE}
[junit4] 2> 2056795 INFO (qtp1875896903-11535) [ x:collection1] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4] 2> 2056795 INFO (qtp1875896903-11535) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{deleteByQuery=*:*} 0 0
[junit4] 2> 2056796 INFO (qtp1875896903-11535) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2056796 INFO (qtp1875896903-11535) [ x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@5059595
[junit4] 2> 2056796 INFO (qtp1875896903-11535) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@718dab44[collection1] main]
[junit4] 2> 2056797 INFO (qtp1875896903-11535) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2056797 INFO (searcherExecutor-5562-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@718dab44[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2056797 INFO (qtp1875896903-11535) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 1
[junit4] 2> 2056810 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@6f0ff173{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2056810 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1407588843
[junit4] 2> 2056811 INFO (coreCloseExecutor-5579-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@629f83ff
[junit4] 2> 2056813 INFO (coreCloseExecutor-5579-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection1
[junit4] 2> 2056814 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 2056815 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4e5de820{/solr,null,UNAVAILABLE}
[junit4] 2> 2056819 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2056829 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@38704be0{/solr,null,AVAILABLE}
[junit4] 2> 2056836 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1957cf43{HTTP/1.1,[http/1.1]}{127.0.0.1:54345}
[junit4] 2> 2056867 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.e.j.s.Server Started @2062001ms
[junit4] 2> 2056867 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_EF0F17C9FF4091C6-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=54345}
[junit4] 2> 2056867 ERROR (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2056867 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 2056868 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 2056868 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2056868 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-09T08:12:53.588Z
[junit4] 2> 2056869 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_EF0F17C9FF4091C6-001/solr-instance-001/solr.xml
[junit4] 2> 2056967 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_EF0F17C9FF4091C6-001/solr-instance-001/.
[junit4] 2> 2056968 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2056992 WARN (coreLoadExecutor-5586-thread-1) [ x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 2056992 INFO (coreLoadExecutor-5586-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2057017 INFO (coreLoadExecutor-5586-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2057024 INFO (coreLoadExecutor-5586-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 2057031 INFO (coreLoadExecutor-5586-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_EF0F17C9FF4091C6-001/solr-instance-001/./collection1
[junit4] 2> 2057031 INFO (coreLoadExecutor-5586-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_EF0F17C9FF4091C6-001/solr-instance-001/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_EF0F17C9FF4091C6-001/solr-instance-001/./collection1/data/]
[junit4] 2> 2057033 INFO (coreLoadExecutor-5586-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=44, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.18221903121906655]
[junit4] 2> 2057136 INFO (coreLoadExecutor-5586-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2057136 INFO (coreLoadExecutor-5586-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2057136 INFO (coreLoadExecutor-5586-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=43, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=99.1337890625, floorSegmentMB=0.34375, forceMergeDeletesPctAllowed=19.395645586126445, segmentsPerTier=20.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 2057137 INFO (coreLoadExecutor-5586-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2d4815cd[collection1] main]
[junit4] 2> 2057137 INFO (coreLoadExecutor-5586-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_EF0F17C9FF4091C6-001/solr-instance-001/collection1/conf
[junit4] 2> 2057138 INFO (coreLoadExecutor-5586-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml
[junit4] 2> 2057138 INFO (coreLoadExecutor-5586-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2057138 INFO (searcherExecutor-5587-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2d4815cd[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2058372 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[0]} 0 0
[junit4] 2> 2058373 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1]} 0 0
[junit4] 2> 2058374 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2]} 0 0
[junit4] 2> 2058376 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[3]} 0 0
[junit4] 2> 2058376 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[4]} 0 0
[junit4] 2> 2058377 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[5]} 0 0
[junit4] 2> 2058378 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[6]} 0 0
[junit4] 2> 2058379 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[7]} 0 0
[junit4] 2> 2058379 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[8]} 0 0
[junit4] 2> 2058380 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[9]} 0 0
[junit4] 2> 2058381 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[10]} 0 0
[junit4] 2> 2058382 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[11]} 0 0
[junit4] 2> 2058383 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[12]} 0 0
[junit4] 2> 2058384 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[13]} 0 0
[junit4] 2> 2058384 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[14]} 0 0
[junit4] 2> 2058385 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[15]} 0 0
[junit4] 2> 2058386 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[16]} 0 0
[junit4] 2> 2058387 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[17]} 0 0
[junit4] 2> 2058387 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[18]} 0 0
[junit4] 2> 2058389 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[19]} 0 0
[junit4] 2> 2058389 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[20]} 0 0
[junit4] 2> 2058390 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[21]} 0 0
[junit4] 2> 2058391 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[22]} 0 0
[junit4] 2> 2058392 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[23]} 0 0
[junit4] 2> 2058392 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[24]} 0 0
[junit4] 2> 2058393 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[25]} 0 0
[junit4] 2> 2058394 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[26]} 0 0
[junit4] 2> 2058395 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[27]} 0 0
[junit4] 2> 2058395 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[28]} 0 0
[junit4] 2> 2058396 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[29]} 0 0
[junit4] 2> 2058397 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[30]} 0 0
[junit4] 2> 2058397 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[31]} 0 0
[junit4] 2> 2058398 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[32]} 0 0
[junit4] 2> 2058399 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[33]} 0 0
[junit4] 2> 2058400 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[34]} 0 0
[junit4] 2> 2058400 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[35]} 0 0
[junit4] 2> 2058401 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[36]} 0 0
[junit4] 2> 2058402 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[37]} 0 0
[junit4] 2> 2058403 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[38]} 0 0
[junit4] 2> 2058404 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[39]} 0 0
[junit4] 2> 2058404 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[40]} 0 0
[junit4] 2> 2058405 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[41]} 0 0
[junit4] 2> 2058406 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[42]} 0 0
[junit4] 2> 2058407 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[43]} 0 0
[junit4] 2> 2058407 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[44]} 0 0
[junit4] 2> 2058408 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[45]} 0 0
[junit4] 2> 2058409 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[46]} 0 0
[junit4] 2> 2058410 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[47]} 0 0
[junit4] 2> 2058410 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[48]} 0 0
[junit4] 2> 2058411 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[49]} 0 0
[junit4] 2> 2058412 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[50]} 0 0
[junit4] 2> 2058413 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[51]} 0 0
[junit4] 2> 2058414 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[52]} 0 0
[junit4] 2> 2058415 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[53]} 0 0
[junit4] 2> 2058415 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[54]} 0 0
[junit4] 2> 2058416 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[55]} 0 0
[junit4] 2> 2058417 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[56]} 0 0
[junit4] 2> 2058418 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[57]} 0 0
[junit4] 2> 2058418 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[58]} 0 0
[junit4] 2> 2058419 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[59]} 0 0
[junit4] 2> 2058420 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[60]} 0 0
[junit4] 2> 2058421 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[61]} 0 0
[junit4] 2> 2058421 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[62]} 0 0
[junit4] 2> 2058422 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[63]} 0 0
[junit4] 2> 2058423 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[64]} 0 0
[junit4] 2> 2058424 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[65]} 0 0
[junit4] 2> 2058425 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[66]} 0 0
[junit4] 2> 2058425 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[67]} 0 0
[junit4] 2> 2058426 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[68]} 0 0
[junit4] 2> 2058427 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[69]} 0 0
[junit4] 2> 2058428 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[70]} 0 0
[junit4] 2> 2058429 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[71]} 0 0
[junit4] 2> 2058429 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[72]} 0 0
[junit4] 2> 2058430 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[73]} 0 0
[junit4] 2> 2058431 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[74]} 0 0
[junit4] 2> 2058432 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[75]} 0 0
[junit4] 2> 2058432 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[76]} 0 0
[junit4] 2> 2058433 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[77]} 0 0
[junit4] 2> 2058434 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[78]} 0 0
[junit4] 2> 2058435 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[79]} 0 0
[junit4] 2> 2058435 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[80]} 0 0
[junit4] 2> 2058436 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[81]} 0 0
[junit4] 2> 2058437 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[82]} 0 0
[junit4] 2> 2058438 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[83]} 0 0
[junit4] 2> 2058439 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[84]} 0 0
[junit4] 2> 2058440 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[85]} 0 0
[junit4] 2> 2058441 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[86]} 0 0
[junit4] 2> 2058442 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[87]} 0 0
[junit4] 2> 2058442 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[88]} 0 0
[junit4] 2> 2058444 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[89]} 0 0
[junit4] 2> 2058445 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[90]} 0 0
[junit4] 2> 2058446 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[91]} 0 0
[junit4] 2> 2058447 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[92]} 0 0
[junit4] 2> 2058448 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[93]} 0 0
[junit4] 2> 2058449 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[94]} 0 0
[junit4] 2> 2058450 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[95]} 0 0
[junit4] 2> 2058451 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[96]} 0 0
[junit4] 2> 2058452 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[97]} 0 0
[junit4] 2> 2058453 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[98]} 0 0
[junit4] 2> 2058453 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[99]} 0 0
[junit4] 2> 2058454 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[100]} 0 0
[junit4] 2> 2058459 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[101]} 0 0
[junit4] 2> 2058460 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[102]} 0 0
[junit4] 2> 2058461 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[103]} 0 0
[junit4] 2> 2058462 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[104]} 0 0
[junit4] 2> 2058463 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[105]} 0 0
[junit4] 2> 2058465 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[106]} 0 0
[junit4] 2> 2058465 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[107]} 0 0
[junit4] 2> 2058466 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[108]} 0 0
[junit4] 2> 2058467 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[109]} 0 0
[junit4] 2> 2058467 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[110]} 0 0
[junit4] 2> 2058468 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[111]} 0 0
[junit4] 2> 2058469 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[112]} 0 0
[junit4] 2> 2058470 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[113]} 0 0
[junit4] 2> 2058470 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[114]} 0 0
[junit4] 2> 2058471 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[115]} 0 0
[junit4] 2> 2058472 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[116]} 0 0
[junit4] 2> 2058473 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[117]} 0 0
[junit4] 2> 2058474 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[118]} 0 0
[junit4] 2> 2058474 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[119]} 0 0
[junit4] 2> 2058475 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[120]} 0 0
[junit4] 2> 2058476 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[121]} 0 0
[junit4] 2> 2058477 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[122]} 0 0
[junit4] 2> 2058477 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[123]} 0 0
[junit4] 2> 2058478 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[124]} 0 0
[junit4] 2> 2058479 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[125]} 0 0
[junit4] 2> 2058480 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[126]} 0 0
[junit4] 2> 2058482 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[127]} 0 1
[junit4] 2> 2058483 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[128]} 0 0
[junit4] 2> 2058486 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[129]} 0 0
[junit4] 2> 2058487 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[130]} 0 0
[junit4] 2> 2058488 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[131]} 0 0
[junit4] 2> 2058489 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[132]} 0 0
[junit4] 2> 2058490 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[133]} 0 0
[junit4] 2> 2058491 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[134]} 0 0
[junit4] 2> 2058492 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[135]} 0 0
[junit4] 2> 2058492 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[136]} 0 0
[junit4] 2> 2058493 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[137]} 0 0
[junit4] 2> 2058494 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[138]} 0 0
[junit4] 2> 2058495 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[139]} 0 0
[junit4] 2> 2058496 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[140]} 0 0
[junit4] 2> 2058497 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[141]} 0 0
[junit4] 2> 2058497 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[142]} 0 0
[junit4] 2> 2058498 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[143]} 0 0
[junit4] 2> 2058498 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[144]} 0 0
[junit4] 2> 2058499 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[145]} 0 0
[junit4] 2> 2058500 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[146]} 0 0
[junit4] 2> 2058501 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[147]} 0 0
[junit4] 2> 2058501 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[148]} 0 0
[junit4] 2> 2058502 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[149]} 0 0
[junit4] 2> 2058503 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[150]} 0 0
[junit4] 2> 2058503 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[151]} 0 0
[junit4] 2> 2058504 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[152]} 0 0
[junit4] 2> 2058505 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[153]} 0 0
[junit4] 2> 2058505 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[154]} 0 0
[junit4] 2> 2058506 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[155]} 0 0
[junit4] 2> 2058507 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[156]} 0 0
[junit4] 2> 2058508 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[157]} 0 0
[junit4] 2> 2058509 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[158]} 0 0
[junit4] 2> 2058509 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[159]} 0 0
[junit4] 2> 2058510 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[160]} 0 0
[junit4] 2> 2058511 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[161]} 0 0
[junit4] 2> 2058511 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[162]} 0 0
[junit4] 2> 2058512 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[163]} 0 0
[junit4] 2> 2058513 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[164]} 0 0
[junit4] 2> 2058513 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[165]} 0 0
[junit4] 2> 2058514 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[166]} 0 0
[junit4] 2> 2058515 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[167]} 0 0
[junit4] 2> 2058515 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[168]} 0 0
[junit4] 2> 2058516 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[169]} 0 0
[junit4] 2> 2058517 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[170]} 0 0
[junit4] 2> 2058517 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[171]} 0 0
[junit4] 2> 2058518 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[172]} 0 0
[junit4] 2> 2058519 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[173]} 0 0
[junit4] 2> 2058520 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[174]} 0 0
[junit4] 2> 2058521 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[175]} 0 0
[junit4] 2> 2058521 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[176]} 0 0
[junit4] 2> 2058522 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[177]} 0 0
[junit4] 2> 2058523 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[178]} 0 0
[junit4] 2> 2058523 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[179]} 0 0
[junit4] 2> 2058524 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[180]} 0 0
[junit4] 2> 2058525 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[181]} 0 0
[junit4] 2> 2058526 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[182]} 0 0
[junit4] 2> 2058526 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[183]} 0 0
[junit4] 2> 2058527 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[184]} 0 0
[junit4] 2> 2058528 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[185]} 0 0
[junit4] 2> 2058528 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[186]} 0 0
[junit4] 2> 2058529 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[187]} 0 0
[junit4] 2> 2058530 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[188]} 0 0
[junit4] 2> 2058530 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[189]} 0 0
[junit4] 2> 2058531 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[190]} 0 0
[junit4] 2> 2058532 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[191]} 0 0
[junit4] 2> 2058533 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[192]} 0 0
[junit4] 2> 2058534 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[193]} 0 0
[junit4] 2> 2058534 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[194]} 0 0
[junit4] 2> 2058535 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[195]} 0 0
[junit4] 2> 2058536 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[196]} 0 0
[junit4] 2> 2058536 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[197]} 0 0
[junit4] 2> 2058537 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[198]} 0 0
[junit4] 2> 2058538 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[199]} 0 0
[junit4] 2> 2058538 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[200]} 0 0
[junit4] 2> 2058539 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[201]} 0 0
[junit4] 2> 2058544 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[202]} 0 0
[junit4] 2> 2058545 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[203]} 0 0
[junit4] 2> 2058546 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[204]} 0 0
[junit4] 2> 2058546 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[205]} 0 0
[junit4] 2> 2058547 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[206]} 0 0
[junit4] 2> 2058549 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[207]} 0 0
[junit4] 2> 2058550 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[208]} 0 0
[junit4] 2> 2058551 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[209]} 0 0
[junit4] 2> 2058552 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[210]} 0 0
[junit4] 2> 2058553 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[211]} 0 0
[junit4] 2> 2058553 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[212]} 0 0
[junit4] 2> 2058554 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[213]} 0 0
[junit4] 2> 2058555 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[214]} 0 0
[junit4] 2> 2058556 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[215]} 0 0
[junit4] 2> 2058557 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[216]} 0 0
[junit4] 2> 2058558 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[217]} 0 0
[junit4] 2> 2058559 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[218]} 0 0
[junit4] 2> 2058559 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[219]} 0 0
[junit4] 2> 2058560 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[220]} 0 0
[junit4] 2> 2058561 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[221]} 0 0
[junit4] 2> 2058561 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[222]} 0 0
[junit4] 2> 2058562 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[223]} 0 0
[junit4] 2> 2058563 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[224]} 0 0
[junit4] 2> 2058564 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[225]} 0 0
[junit4] 2> 2058565 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[226]} 0 0
[junit4] 2> 2058565 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[227]} 0 0
[junit4] 2> 2058566 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[228]} 0 0
[junit4] 2> 2058567 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[229]} 0 0
[junit4] 2> 2058568 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[230]} 0 0
[junit4] 2> 2058569 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[231]} 0 0
[junit4] 2> 2058570 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[232]} 0 0
[junit4] 2> 2058570 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[233]} 0 0
[junit4] 2> 2058571 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[234]} 0 0
[junit4] 2> 2058572 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[235]} 0 0
[junit4] 2> 2058573 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[236]} 0 0
[junit4] 2> 2058574 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[237]} 0 0
[junit4] 2> 2058575 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[238]} 0 0
[junit4] 2> 2058576 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[239]} 0 0
[junit4] 2> 2058577 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[240]} 0 0
[junit4] 2> 2058578 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[241]} 0 0
[junit4] 2> 2058578 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[242]} 0 0
[junit4] 2> 2058579 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[243]} 0 0
[junit4] 2> 2058580 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[244]} 0 0
[junit4] 2> 2058581 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[245]} 0 0
[junit4] 2> 2058581 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[246]} 0 0
[junit4] 2> 2058582 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[247]} 0 0
[junit4] 2> 2058583 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[248]} 0 0
[junit4] 2> 2058583 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[249]} 0 0
[junit4] 2> 2058585 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[250]} 0 0
[junit4] 2> 2058586 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[251]} 0 0
[junit4] 2> 2058587 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[252]} 0 0
[junit4] 2> 2058587 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[253]} 0 0
[junit4] 2> 2058588 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[254]} 0 0
[junit4] 2> 2058590 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[255]} 0 1
[junit4] 2> 2058592 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[256]} 0 0
[junit4] 2> 2058596 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[257]} 0 0
[junit4] 2> 2058597 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[258]} 0 0
[junit4] 2> 2058599 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[259]} 0 0
[junit4] 2> 2058599 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[260]} 0 0
[junit4] 2> 2058600 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[261]} 0 0
[junit4] 2> 2058601 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[262]} 0 0
[junit4] 2> 2058602 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[263]} 0 0
[junit4] 2> 2058603 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[264]} 0 0
[junit4] 2> 2058604 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[265]} 0 0
[junit4] 2> 2058605 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[266]} 0 0
[junit4] 2> 2058606 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[267]} 0 0
[junit4] 2> 2058606 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[268]} 0 0
[junit4] 2> 2058607 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[269]} 0 0
[junit4] 2> 2058608 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[270]} 0 0
[junit4] 2> 2058609 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[271]} 0 0
[junit4] 2> 2058609 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[272]} 0 0
[junit4] 2> 2058610 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[273]} 0 0
[junit4] 2> 2058611 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[274]} 0 0
[junit4] 2> 2058612 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[275]} 0 0
[junit4] 2> 2058612 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[276]} 0 0
[junit4] 2> 2058613 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[277]} 0 0
[junit4] 2> 2058614 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[278]} 0 0
[junit4] 2> 2058615 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[279]} 0 0
[junit4] 2> 2058616 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[280]} 0 0
[junit4] 2> 2058617 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[281]} 0 0
[junit4] 2> 2058618 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[282]} 0 0
[junit4] 2> 2058619 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[283]} 0 0
[junit4] 2> 2058619 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[284]} 0 0
[junit4] 2> 2058620 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[285]} 0 0
[junit4] 2> 2058621 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[286]} 0 0
[junit4] 2> 2058622 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[287]} 0 0
[junit4] 2> 2058623 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[288]} 0 0
[junit4] 2> 2058624 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[289]} 0 0
[junit4] 2> 2058624 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[290]} 0 0
[junit4] 2> 2058625 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[291]} 0 0
[junit4] 2> 2058626 INFO (qtp1741501237-11576) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[292]} 0 0
[junit4] 2> 2058627 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[293]} 0 0
[junit4] 2> 2058628 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[294]} 0 0
[junit4] 2> 2058628 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[295]} 0 0
[junit4] 2> 2058629 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[296]} 0 0
[junit4] 2> 2058630 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[297]} 0 0
[junit4] 2> 2058630 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[298]} 0 0
[junit4] 2> 2058631 INFO (qtp1741501237-11573) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[299]} 0 0
[junit4] 2> 2058632 INFO (qtp1741501237-11570) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[300]} 0 0
[junit4] 2> 2058633 INFO (qtp1741501237-11572) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[301]} 0 0
[junit4] 2> 2058634 INFO (qtp1741501237-11577) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[302]} 0 0
[junit4] 2> 2058635 INFO (qtp1741501237-11575) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[303]} 0 0
[junit4] 2> 2058636 INFO (qtp1741501237-11574) [ x:collection1] o.a.s.u.
[...truncated too long message...]
SSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 2179595 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 2179599 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@65be1952[collection1] main]
[junit4] 2> 2179599 INFO (searcherExecutor-6198-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@65be1952[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):c3) Uninverting(_1(7.0.0):c3)))}
[junit4] 2> 2179600 INFO (qtp516796158-12547) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={wait=true&masterUrl=http://127.0.0.1:54600/solr/collection1/replication&command=fetchindex} status=0 QTime=24
[junit4] 2> 2179602 INFO (qtp516796158-12548) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/select params={q=*:*&sort=id+desc&wt=javabin&version=2} hits=6 status=0 QTime=0
[junit4] 2> 2179603 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.h.TestReplicationHandler Waited for 0ms and found 6 docs
[junit4] 2> 2179604 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[EF0F17C9FF4091C6]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@77a0ca03{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2179604 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=652199158
[junit4] 2> 2179605 INFO (coreCloseExecutor-6217-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@12cd943b
[junit4] 2> 2179606 INFO (coreCloseExecutor-6217-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection1
[junit4] 2> 2179607 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 2179607 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[EF0F17C9FF4091C6]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@35618be7{/solr,null,UNAVAILABLE}
[junit4] 2> 2179608 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.SolrTestCaseJ4 ###Ending doTestRepeater
[junit4] 2> 2179610 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[EF0F17C9FF4091C6]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@7ee7f7a9{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2179610 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=593731299
[junit4] 2> 2179611 INFO (coreCloseExecutor-6219-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@76c49e11
[junit4] 2> 2179612 INFO (coreCloseExecutor-6219-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection1
[junit4] 2> 2179613 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 2179613 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[EF0F17C9FF4091C6]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2c25996f{/solr,null,UNAVAILABLE}
[junit4] 2> 2179616 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[EF0F17C9FF4091C6]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@2e62b753{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2179616 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=517719592
[junit4] 2> 2179617 INFO (coreCloseExecutor-6221-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@4b762852
[junit4] 2> 2179617 INFO (coreCloseExecutor-6221-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection1
[junit4] 2> 2179618 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[EF0F17C9FF4091C6]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 2179618 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[EF0F17C9FF4091C6]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1636e55f{/solr,null,UNAVAILABLE}
[junit4] 2> 2179621 INFO (SUITE-TestReplicationHandler-seed#[EF0F17C9FF4091C6]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_EF0F17C9FF4091C6-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {name=PostingsFormat(name=MockRandom), id=PostingsFormat(name=MockRandom), newname=PostingsFormat(name=MockRandom)}, docValues:{}, maxPointsInLeafNode=2031, maxMBSortInHeap=6.997661629203152, sim=RandomSimilarity(queryNorm=true): {}, locale=nl-NL, timezone=America/Goose_Bay
[junit4] 2> NOTE: Mac OS X 10.11.6 x86_64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=3,threads=1,free=191821688,total=447385600
[junit4] 2> NOTE: All tests run in this JVM: [StatsComponentTest, TestSchemaVersionResource, TestSuggestSpellingConverter, TestLMDirichletSimilarityFactory, LeaderInitiatedRecoveryOnCommitTest, SuggestComponentContextFilterQueryTest, MinimalSchemaTest, TestLegacyFieldCache, TestSQLHandler, TestMacroExpander, ConcurrentDeleteAndCreateCollectionTest, DistributedTermsComponentTest, CoreAdminCreateDiscoverTest, TestSolrQueryParser, TestRecovery, TestAnalyzedSuggestions, ActionThrottleTest, TestHighFrequencyDictionaryFactory, TestOmitPositions, FieldAnalysisRequestHandlerTest, TestDFRSimilarityFactory, OverseerModifyCollectionTest, BadIndexSchemaTest, AnalyticsQueryTest, HighlighterConfigTest, TestAuthorizationFramework, MigrateRouteKeyTest, BasicFunctionalityTest, TestNumericTerms64, JSONWriterTest, TestUnifiedSolrHighlighter, SpellingQueryConverterTest, CacheHeaderTest, UtilsToolTest, TestJsonRequest, EchoParamsTest, TestBinaryResponseWriter, JsonLoaderTest, TestDistributedGrouping, AssignTest, TolerantUpdateProcessorTest, TestLegacyNumericRangeQueryBuilder, TestComplexPhraseLeadingWildcard, TestBadConfig, TestDocSet, TestStressInPlaceUpdates, TestSolrCloudWithDelegationTokens, CSVRequestHandlerTest, ClassificationUpdateProcessorIntegrationTest, UpdateRequestProcessorFactoryTest, OverseerStatusTest, TestLRUStatsCache, ReturnFieldsTest, TestWriterPerf, RuleEngineTest, SpellCheckComponentTest, AddBlockUpdateTest, TestHdfsCloudBackupRestore, TestDynamicFieldResource, ReplicationFactorTest, IndexSchemaTest, TestRealTimeGet, TestSolrCoreProperties, CollectionStateFormat2Test, DocValuesNotIndexedTest, ResponseLogComponentTest, EnumFieldTest, TestCloudInspectUtil, VMParamsZkACLAndCredentialsProvidersTest, TestFieldCacheReopen, MergeStrategyTest, SuggesterTSTTest, TestJettySolrRunner, CdcrReplicationDistributedZkTest, SolrInfoMBeanTest, TestSolrIndexConfig, TestSolr4Spatial2, UpdateParamsTest, TestSolrXml, RAMDirectoryFactoryTest, HdfsNNFailoverTest, LoggingHandlerTest, HdfsSyncSliceTest, RequestHandlersTest, CurrencyFieldOpenExchangeTest, TestImplicitCoreProperties, SolrCoreCheckLockOnStartupTest, TestUseDocValuesAsStored2, TestReversedWildcardFilterFactory, SortByFunctionTest, SimpleMLTQParserTest, DefaultValueUpdateProcessorTest, HLLSerializationTest, TestSearchPerf, HdfsTlogReplayBufferedWhileIndexingTest, TestManagedSynonymFilterFactory, TestDocBasedVersionConstraints, ConvertedLegacyTest, TermVectorComponentDistributedTest, TestFuzzyAnalyzedSuggestions, CoreMergeIndexesAdminHandlerTest, TestChildDocTransformer, TestUtils, TestPointFields, TestConfigSetImmutable, TestFileDictionaryLookup, TestLMJelinekMercerSimilarityFactory, SuggesterWFSTTest, TestConfig, DebugComponentTest, TestDFISimilarityFactory, LeaderElectionIntegrationTest, BaseCdcrDistributedZkTest, TestCoreDiscovery, TestMissingGroups, InfixSuggestersTest, SolrCmdDistributorTest, DeleteLastCustomShardedReplicaTest, TestRuleBasedAuthorizationPlugin, TestJmxMonitoredMap, SegmentsInfoRequestHandlerTest, BasicDistributedZk2Test, DistanceUnitsTest, RemoteQueryErrorTest, PeerSyncReplicationTest, DistribDocExpirationUpdateProcessorTest, TestDownShardTolerantSearch, TestReloadAndDeleteDocs, TestSchemaResource, TestQuerySenderListener, DirectoryFactoryTest, TestDeleteCollectionOnDownNodes, TestQueryUtils, TestFieldTypeResource, BlockJoinFacetDistribTest, TestSolrDeletionPolicy2, BlobRepositoryCloudTest, ShardRoutingCustomTest, ResponseHeaderTest, VersionInfoTest, ExternalFileFieldSortTest, TestSolrCloudWithSecureImpersonation, TestPHPSerializedResponseWriter, TestHashQParserPlugin, HttpPartitionTest, TestPerFieldSimilarityWithDefaultOverride, TestSolrFieldCacheMBean, TestBackupRepositoryFactory, TestSimpleQParserPlugin, TestJsonFacetRefinement, TestLeaderElectionZkExpiry, BooleanFieldTest, TestInPlaceUpdatesStandalone, NotRequiredUniqueKeyTest, TestExactSharedStatsCache, TestCustomStream, ChaosMonkeySafeLeaderTest, ExplicitHLLTest, TestManagedResource, DistributedQueueTest, PreAnalyzedUpdateProcessorTest, TestSolr4Spatial, SparseHLLTest, TestSystemCollAutoCreate, DistanceFunctionTest, SharedFSAutoReplicaFailoverTest, BasicDistributedZkTest, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest, SyncSliceTest, OpenCloseCoreStressTest, OverseerTest, ShardRoutingTest, BasicZkTest, FullSolrCloudDistribCmdsTest, TestReplicationHandler]
[junit4] Completed [356/693 (1!)] on J1 in 126.53s, 16 tests, 1 error, 1 skipped <<< FAILURES!
[...truncated 63565 lines...]
[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3820
- Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3820/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC
2 tests failed.
FAILED: org.apache.solr.cloud.CustomCollectionTest.testCustomCollectionsAPI
Error Message:
Could not find collection : implicitcoll
Stack Trace:
org.apache.solr.common.SolrException: Could not find collection : implicitcoll
at __randomizedtesting.SeedInfo.seed([D62FC493DF153A9E:BCCE4AF8E28F8CE6]:0)
at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:194)
at org.apache.solr.cloud.SolrCloudTestCase.getCollectionState(SolrCloudTestCase.java:245)
at org.apache.solr.cloud.CustomCollectionTest.testCustomCollectionsAPI(CustomCollectionTest.java:68)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.PeerSyncReplicationTest.test
Error Message:
timeout waiting to see all nodes active
Stack Trace:
java.lang.AssertionError: timeout waiting to see all nodes active
at __randomizedtesting.SeedInfo.seed([D62FC493DF153A9E:5E7BFB4971E95766]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.PeerSyncReplicationTest.waitTillNodesActive(PeerSyncReplicationTest.java:326)
at org.apache.solr.cloud.PeerSyncReplicationTest.bringUpDeadNodeAndEnsureNoReplication(PeerSyncReplicationTest.java:277)
at org.apache.solr.cloud.PeerSyncReplicationTest.forceNodeFailureAndDoPeerSync(PeerSyncReplicationTest.java:259)
at org.apache.solr.cloud.PeerSyncReplicationTest.test(PeerSyncReplicationTest.java:138)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11321 lines...]
[junit4] Suite: org.apache.solr.cloud.CustomCollectionTest
[junit4] 2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/init-core-data-001
[junit4] 2> 894620 INFO (SUITE-CustomCollectionTest-seed#[D62FC493DF153A9E]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 894625 INFO (SUITE-CustomCollectionTest-seed#[D62FC493DF153A9E]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN) w/ MAC_OS_X supressed clientAuth
[junit4] 2> 894629 INFO (SUITE-CustomCollectionTest-seed#[D62FC493DF153A9E]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001
[junit4] 2> 894629 INFO (SUITE-CustomCollectionTest-seed#[D62FC493DF153A9E]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 894630 INFO (Thread-1289) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 894630 INFO (Thread-1289) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 894732 INFO (SUITE-CustomCollectionTest-seed#[D62FC493DF153A9E]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:64299
[junit4] 2> 895111 INFO (jetty-launcher-966-thread-1) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 895111 INFO (jetty-launcher-966-thread-2) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 895112 INFO (jetty-launcher-966-thread-4) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 895114 INFO (jetty-launcher-966-thread-3) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 895123 INFO (jetty-launcher-966-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1f769c0d{/solr,null,AVAILABLE}
[junit4] 2> 895124 INFO (jetty-launcher-966-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7031d0bc{/solr,null,AVAILABLE}
[junit4] 2> 895125 INFO (jetty-launcher-966-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3f18912{/solr,null,AVAILABLE}
[junit4] 2> 895126 INFO (jetty-launcher-966-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@29401ee5{/solr,null,AVAILABLE}
[junit4] 2> 895127 INFO (jetty-launcher-966-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@58bc8def{HTTP/1.1,[http/1.1]}{127.0.0.1:64303}
[junit4] 2> 895127 INFO (jetty-launcher-966-thread-3) [ ] o.e.j.s.Server Started @901021ms
[junit4] 2> 895127 INFO (jetty-launcher-966-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4b06652f{HTTP/1.1,[http/1.1]}{127.0.0.1:64302}
[junit4] 2> 895127 INFO (jetty-launcher-966-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=64303}
[junit4] 2> 895127 INFO (jetty-launcher-966-thread-2) [ ] o.e.j.s.Server Started @901021ms
[junit4] 2> 895127 INFO (jetty-launcher-966-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=64302}
[junit4] 2> 895127 ERROR (jetty-launcher-966-thread-2) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 895127 ERROR (jetty-launcher-966-thread-3) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 895127 INFO (jetty-launcher-966-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2b94fabe{HTTP/1.1,[http/1.1]}{127.0.0.1:64301}
[junit4] 2> 895128 INFO (jetty-launcher-966-thread-4) [ ] o.e.j.s.Server Started @901021ms
[junit4] 2> 895128 INFO (jetty-launcher-966-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1847eb5c{HTTP/1.1,[http/1.1]}{127.0.0.1:64304}
[junit4] 2> 895128 INFO (jetty-launcher-966-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 895128 INFO (jetty-launcher-966-thread-1) [ ] o.e.j.s.Server Started @901022ms
[junit4] 2> 895128 INFO (jetty-launcher-966-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 895128 INFO (jetty-launcher-966-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=64304}
[junit4] 2> 895128 INFO (jetty-launcher-966-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 895128 INFO (jetty-launcher-966-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 895128 INFO (jetty-launcher-966-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-08T19:52:23.666Z
[junit4] 2> 895128 INFO (jetty-launcher-966-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 895128 INFO (jetty-launcher-966-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 895128 ERROR (jetty-launcher-966-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 895128 INFO (jetty-launcher-966-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-08T19:52:23.666Z
[junit4] 2> 895128 INFO (jetty-launcher-966-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=64301}
[junit4] 2> 895129 ERROR (jetty-launcher-966-thread-4) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 895130 INFO (jetty-launcher-966-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 895130 INFO (jetty-launcher-966-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 895131 INFO (jetty-launcher-966-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 895131 INFO (jetty-launcher-966-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 895131 INFO (jetty-launcher-966-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 895131 INFO (jetty-launcher-966-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 895131 INFO (jetty-launcher-966-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-08T19:52:23.669Z
[junit4] 2> 895131 INFO (jetty-launcher-966-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-08T19:52:23.669Z
[junit4] 2> 895143 INFO (jetty-launcher-966-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 895144 INFO (jetty-launcher-966-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 895144 INFO (jetty-launcher-966-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 895144 INFO (jetty-launcher-966-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 895181 INFO (jetty-launcher-966-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:64299/solr
[junit4] 2> 895181 INFO (jetty-launcher-966-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:64299/solr
[junit4] 2> 895182 INFO (jetty-launcher-966-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:64299/solr
[junit4] 2> 895187 INFO (jetty-launcher-966-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:64299/solr
[junit4] 2> 895264 INFO (jetty-launcher-966-thread-3) [n:127.0.0.1:64303_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:64303_solr
[junit4] 2> 895268 INFO (jetty-launcher-966-thread-2) [n:127.0.0.1:64302_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64302_solr
[junit4] 2> 895268 INFO (jetty-launcher-966-thread-1) [n:127.0.0.1:64304_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64304_solr
[junit4] 2> 895268 INFO (jetty-launcher-966-thread-4) [n:127.0.0.1:64301_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64301_solr
[junit4] 2> 895269 INFO (jetty-launcher-966-thread-3) [n:127.0.0.1:64303_solr ] o.a.s.c.Overseer Overseer (id=97424739086958601-127.0.0.1:64303_solr-n_0000000000) starting
[junit4] 2> 895273 INFO (zkCallback-984-thread-1-processing-n:127.0.0.1:64303_solr) [n:127.0.0.1:64303_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 895274 INFO (zkCallback-985-thread-1-processing-n:127.0.0.1:64301_solr) [n:127.0.0.1:64301_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 895275 INFO (zkCallback-986-thread-1-processing-n:127.0.0.1:64302_solr) [n:127.0.0.1:64302_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 895275 INFO (zkCallback-987-thread-1-processing-n:127.0.0.1:64304_solr) [n:127.0.0.1:64304_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 895321 INFO (jetty-launcher-966-thread-3) [n:127.0.0.1:64303_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64303_solr
[junit4] 2> 895324 INFO (zkCallback-985-thread-1-processing-n:127.0.0.1:64301_solr) [n:127.0.0.1:64301_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 895324 INFO (zkCallback-984-thread-1-processing-n:127.0.0.1:64303_solr) [n:127.0.0.1:64303_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 895325 INFO (zkCallback-987-thread-1-processing-n:127.0.0.1:64304_solr) [n:127.0.0.1:64304_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 895325 INFO (zkCallback-986-thread-1-processing-n:127.0.0.1:64302_solr) [n:127.0.0.1:64302_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 895840 INFO (jetty-launcher-966-thread-2) [n:127.0.0.1:64302_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node2/.
[junit4] 2> 895841 INFO (jetty-launcher-966-thread-1) [n:127.0.0.1:64304_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node1/.
[junit4] 2> 895842 INFO (jetty-launcher-966-thread-3) [n:127.0.0.1:64303_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node3/.
[junit4] 2> 895842 INFO (jetty-launcher-966-thread-4) [n:127.0.0.1:64301_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node4/.
[junit4] 2> 896561 INFO (SUITE-CustomCollectionTest-seed#[D62FC493DF153A9E]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 896563 INFO (SUITE-CustomCollectionTest-seed#[D62FC493DF153A9E]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:64299/solr ready
[junit4] 2> 896610 INFO (TEST-CustomCollectionTest.testCreateShardRepFactor-seed#[D62FC493DF153A9E]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCreateShardRepFactor
[junit4] 2> 896612 INFO (TEST-CustomCollectionTest.testCreateShardRepFactor-seed#[D62FC493DF153A9E]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 896618 INFO (qtp1885666094-5479) [n:127.0.0.1:64301_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params shards=a,b&replicationFactor=1&collection.configName=conf&name=testCreateShardRepFactor&router.name=implicit&action=CREATE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 896622 INFO (OverseerThreadFactory-2585-thread-1-processing-n:127.0.0.1:64303_solr) [n:127.0.0.1:64303_solr ] o.a.s.c.CreateCollectionCmd Create collection testCreateShardRepFactor
[junit4] 2> 896737 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=testCreateShardRepFactor_b_replica1&action=CREATE&numShards=2&collection=testCreateShardRepFactor&shard=b&wt=javabin&version=2
[junit4] 2> 896737 INFO (qtp552936436-5488) [n:127.0.0.1:64302_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=testCreateShardRepFactor_a_replica1&action=CREATE&numShards=2&collection=testCreateShardRepFactor&shard=a&wt=javabin&version=2
[junit4] 2> 896849 INFO (zkCallback-986-thread-1-processing-n:127.0.0.1:64302_solr) [n:127.0.0.1:64302_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 896849 INFO (zkCallback-985-thread-1-processing-n:127.0.0.1:64301_solr) [n:127.0.0.1:64301_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 897756 INFO (qtp552936436-5488) [n:127.0.0.1:64302_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 897756 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 897778 INFO (qtp552936436-5488) [n:127.0.0.1:64302_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.s.IndexSchema [testCreateShardRepFactor_a_replica1] Schema name=cloud-dynamic
[junit4] 2> 897778 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.s.IndexSchema [testCreateShardRepFactor_b_replica1] Schema name=cloud-dynamic
[junit4] 2> 897826 WARN (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.s.IndexSchema [testCreateShardRepFactor_b_replica1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 897826 WARN (qtp552936436-5488) [n:127.0.0.1:64302_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.s.IndexSchema [testCreateShardRepFactor_a_replica1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 897829 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 897830 INFO (qtp552936436-5488) [n:127.0.0.1:64302_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 897832 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.c.CoreContainer Creating SolrCore 'testCreateShardRepFactor_b_replica1' using configuration from collection testCreateShardRepFactor
[junit4] 2> 897832 INFO (qtp552936436-5488) [n:127.0.0.1:64302_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.c.CoreContainer Creating SolrCore 'testCreateShardRepFactor_a_replica1' using configuration from collection testCreateShardRepFactor
[junit4] 2> 897832 INFO (qtp552936436-5488) [n:127.0.0.1:64302_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.c.SolrCore [[testCreateShardRepFactor_a_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node2/testCreateShardRepFactor_a_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node2/./testCreateShardRepFactor_a_replica1/data/]
[junit4] 2> 897832 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.c.SolrCore [[testCreateShardRepFactor_b_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node4/testCreateShardRepFactor_b_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node4/./testCreateShardRepFactor_b_replica1/data/]
[junit4] 2> 897912 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 897912 INFO (qtp552936436-5488) [n:127.0.0.1:64302_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 897912 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 897912 INFO (qtp552936436-5488) [n:127.0.0.1:64302_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 897914 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 897914 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 897914 INFO (qtp552936436-5488) [n:127.0.0.1:64302_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 897914 INFO (qtp552936436-5488) [n:127.0.0.1:64302_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 897915 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5d05bcc8[testCreateShardRepFactor_b_replica1] main]
[junit4] 2> 897916 INFO (qtp552936436-5488) [n:127.0.0.1:64302_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@29c8fb6c[testCreateShardRepFactor_a_replica1] main]
[junit4] 2> 897917 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 897917 INFO (qtp552936436-5488) [n:127.0.0.1:64302_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 897918 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 897918 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 897918 INFO (qtp552936436-5488) [n:127.0.0.1:64302_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 897918 INFO (qtp552936436-5488) [n:127.0.0.1:64302_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 897918 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558795828808646656
[junit4] 2> 897918 INFO (searcherExecutor-2594-thread-1-processing-n:127.0.0.1:64301_solr x:testCreateShardRepFactor_b_replica1 s:b c:testCreateShardRepFactor) [n:127.0.0.1:64301_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.c.SolrCore [testCreateShardRepFactor_b_replica1] Registered new searcher Searcher@5d05bcc8[testCreateShardRepFactor_b_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 897918 INFO (qtp552936436-5488) [n:127.0.0.1:64302_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558795828808646656
[junit4] 2> 897918 INFO (searcherExecutor-2595-thread-1-processing-n:127.0.0.1:64302_solr x:testCreateShardRepFactor_a_replica1 s:a c:testCreateShardRepFactor) [n:127.0.0.1:64302_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.c.SolrCore [testCreateShardRepFactor_a_replica1] Registered new searcher Searcher@29c8fb6c[testCreateShardRepFactor_a_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 897929 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 897929 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 897929 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:64301/solr/testCreateShardRepFactor_b_replica1/
[junit4] 2> 897929 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 897929 INFO (qtp552936436-5488) [n:127.0.0.1:64302_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 897929 INFO (qtp552936436-5488) [n:127.0.0.1:64302_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 897929 INFO (qtp552936436-5488) [n:127.0.0.1:64302_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:64302/solr/testCreateShardRepFactor_a_replica1/
[junit4] 2> 897929 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.c.SyncStrategy http://127.0.0.1:64301/solr/testCreateShardRepFactor_b_replica1/ has no replicas
[junit4] 2> 897929 INFO (qtp552936436-5488) [n:127.0.0.1:64302_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 897929 INFO (qtp552936436-5488) [n:127.0.0.1:64302_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.c.SyncStrategy http://127.0.0.1:64302/solr/testCreateShardRepFactor_a_replica1/ has no replicas
[junit4] 2> 897937 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:64301/solr/testCreateShardRepFactor_b_replica1/ b
[junit4] 2> 897937 INFO (qtp552936436-5488) [n:127.0.0.1:64302_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:64302/solr/testCreateShardRepFactor_a_replica1/ a
[junit4] 2> 898044 INFO (zkCallback-985-thread-1-processing-n:127.0.0.1:64301_solr) [n:127.0.0.1:64301_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 898044 INFO (zkCallback-986-thread-1-processing-n:127.0.0.1:64302_solr) [n:127.0.0.1:64302_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 898090 INFO (qtp552936436-5488) [n:127.0.0.1:64302_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 898090 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 898096 INFO (qtp552936436-5488) [n:127.0.0.1:64302_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=testCreateShardRepFactor_a_replica1&action=CREATE&numShards=2&collection=testCreateShardRepFactor&shard=a&wt=javabin&version=2} status=0 QTime=1359
[junit4] 2> 898097 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=testCreateShardRepFactor_b_replica1&action=CREATE&numShards=2&collection=testCreateShardRepFactor&shard=b&wt=javabin&version=2} status=0 QTime=1360
[junit4] 2> 898103 INFO (qtp1885666094-5479) [n:127.0.0.1:64301_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 898201 INFO (zkCallback-985-thread-1-processing-n:127.0.0.1:64301_solr) [n:127.0.0.1:64301_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 898201 INFO (zkCallback-986-thread-1-processing-n:127.0.0.1:64302_solr) [n:127.0.0.1:64302_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 898626 INFO (OverseerCollectionConfigSetProcessor-97424739086958601-127.0.0.1:64303_solr-n_0000000000) [n:127.0.0.1:64303_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 899103 INFO (qtp1885666094-5479) [n:127.0.0.1:64301_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={shards=a,b&replicationFactor=1&collection.configName=conf&name=testCreateShardRepFactor&router.name=implicit&action=CREATE&wt=javabin&version=2} status=0 QTime=2485
[junit4] 2> 899111 INFO (qtp1885666094-5482) [n:127.0.0.1:64301_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createshard with params action=CREATESHARD&collection=testCreateShardRepFactor&shard=x&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 899115 INFO (OverseerThreadFactory-2585-thread-2-processing-n:127.0.0.1:64303_solr) [n:127.0.0.1:64303_solr ] o.a.s.c.CreateShardCmd Create shard invoked: {
[junit4] 2> "collection":"testCreateShardRepFactor",
[junit4] 2> "shard":"x",
[junit4] 2> "operation":"createshard"}
[junit4] 2> 899226 INFO (zkCallback-986-thread-1-processing-n:127.0.0.1:64302_solr) [n:127.0.0.1:64302_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 899227 INFO (zkCallback-985-thread-1-processing-n:127.0.0.1:64301_solr) [n:127.0.0.1:64301_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 899324 INFO (OverseerThreadFactory-2585-thread-2-processing-n:127.0.0.1:64303_solr) [n:127.0.0.1:64303_solr ] o.a.s.c.CreateShardCmd Creating shard testCreateShardRepFactor_x_replica1 as part of slice x of collection testCreateShardRepFactor on 127.0.0.1:64304_solr
[junit4] 2> 899342 INFO (qtp1474850440-5504) [n:127.0.0.1:64304_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&name=testCreateShardRepFactor_x_replica1&action=CREATE&numShards=1&collection=testCreateShardRepFactor&shard=x&wt=javabin&version=2
[junit4] 2> 899458 INFO (zkCallback-985-thread-1-processing-n:127.0.0.1:64301_solr) [n:127.0.0.1:64301_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 899458 INFO (zkCallback-986-thread-1-processing-n:127.0.0.1:64302_solr) [n:127.0.0.1:64302_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 899458 INFO (zkCallback-987-thread-1-processing-n:127.0.0.1:64304_solr) [n:127.0.0.1:64304_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 900368 INFO (qtp1474850440-5504) [n:127.0.0.1:64304_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 900377 INFO (qtp1474850440-5504) [n:127.0.0.1:64304_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.s.IndexSchema [testCreateShardRepFactor_x_replica1] Schema name=cloud-dynamic
[junit4] 2> 900423 WARN (qtp1474850440-5504) [n:127.0.0.1:64304_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.s.IndexSchema [testCreateShardRepFactor_x_replica1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 900424 INFO (qtp1474850440-5504) [n:127.0.0.1:64304_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 900427 INFO (qtp1474850440-5504) [n:127.0.0.1:64304_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.c.CoreContainer Creating SolrCore 'testCreateShardRepFactor_x_replica1' using configuration from collection testCreateShardRepFactor
[junit4] 2> 900427 INFO (qtp1474850440-5504) [n:127.0.0.1:64304_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.c.SolrCore [[testCreateShardRepFactor_x_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node1/testCreateShardRepFactor_x_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node1/./testCreateShardRepFactor_x_replica1/data/]
[junit4] 2> 900503 INFO (qtp1474850440-5504) [n:127.0.0.1:64304_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 900503 INFO (qtp1474850440-5504) [n:127.0.0.1:64304_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 900505 INFO (qtp1474850440-5504) [n:127.0.0.1:64304_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 900505 INFO (qtp1474850440-5504) [n:127.0.0.1:64304_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 900506 INFO (qtp1474850440-5504) [n:127.0.0.1:64304_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1c63195b[testCreateShardRepFactor_x_replica1] main]
[junit4] 2> 900508 INFO (qtp1474850440-5504) [n:127.0.0.1:64304_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 900509 INFO (qtp1474850440-5504) [n:127.0.0.1:64304_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 900509 INFO (qtp1474850440-5504) [n:127.0.0.1:64304_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 900509 INFO (qtp1474850440-5504) [n:127.0.0.1:64304_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558795831525507072
[junit4] 2> 900509 INFO (searcherExecutor-2604-thread-1-processing-n:127.0.0.1:64304_solr x:testCreateShardRepFactor_x_replica1 s:x c:testCreateShardRepFactor) [n:127.0.0.1:64304_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.c.SolrCore [testCreateShardRepFactor_x_replica1] Registered new searcher Searcher@1c63195b[testCreateShardRepFactor_x_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 900520 INFO (qtp1474850440-5504) [n:127.0.0.1:64304_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 900520 INFO (qtp1474850440-5504) [n:127.0.0.1:64304_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 900520 INFO (qtp1474850440-5504) [n:127.0.0.1:64304_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:64304/solr/testCreateShardRepFactor_x_replica1/
[junit4] 2> 900520 INFO (qtp1474850440-5504) [n:127.0.0.1:64304_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 900520 INFO (qtp1474850440-5504) [n:127.0.0.1:64304_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.c.SyncStrategy http://127.0.0.1:64304/solr/testCreateShardRepFactor_x_replica1/ has no replicas
[junit4] 2> 900526 INFO (qtp1474850440-5504) [n:127.0.0.1:64304_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:64304/solr/testCreateShardRepFactor_x_replica1/ x
[junit4] 2> 900632 INFO (zkCallback-985-thread-1-processing-n:127.0.0.1:64301_solr) [n:127.0.0.1:64301_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 900632 INFO (zkCallback-987-thread-1-processing-n:127.0.0.1:64304_solr) [n:127.0.0.1:64304_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 900633 INFO (zkCallback-986-thread-1-processing-n:127.0.0.1:64302_solr) [n:127.0.0.1:64302_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 900680 INFO (qtp1474850440-5504) [n:127.0.0.1:64304_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 900684 INFO (qtp1474850440-5504) [n:127.0.0.1:64304_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&name=testCreateShardRepFactor_x_replica1&action=CREATE&numShards=1&collection=testCreateShardRepFactor&shard=x&wt=javabin&version=2} status=0 QTime=1341
[junit4] 2> 900685 INFO (OverseerThreadFactory-2585-thread-2-processing-n:127.0.0.1:64303_solr) [n:127.0.0.1:64303_solr ] o.a.s.c.CreateShardCmd Finished create command on all shards for collection: testCreateShardRepFactor
[junit4] 2> 900689 INFO (qtp1885666094-5482) [n:127.0.0.1:64301_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=CREATESHARD&collection=testCreateShardRepFactor&shard=x&wt=javabin&version=2} status=0 QTime=1578
[junit4] 2> 900696 INFO (TEST-CustomCollectionTest.testCreateShardRepFactor-seed#[D62FC493DF153A9E]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testCreateShardRepFactor
[junit4] 2> 900746 INFO (TEST-CustomCollectionTest.testRouteFieldForImplicitRouter-seed#[D62FC493DF153A9E]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testRouteFieldForImplicitRouter
[junit4] 2> 900748 INFO (TEST-CustomCollectionTest.testRouteFieldForImplicitRouter-seed#[D62FC493DF153A9E]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 900751 INFO (qtp1885666094-5483) [n:127.0.0.1:64301_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=testCreateShardRepFactor&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 900756 INFO (OverseerCollectionConfigSetProcessor-97424739086958601-127.0.0.1:64303_solr-n_0000000000) [n:127.0.0.1:64303_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 900756 INFO (OverseerThreadFactory-2585-thread-3-processing-n:127.0.0.1:64303_solr) [n:127.0.0.1:64303_solr ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
[junit4] 2> 900759 INFO (qtp552936436-5489) [n:127.0.0.1:64302_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.testCreateShardRepFactor.a.replica1
[junit4] 2> 900759 INFO (qtp1885666094-5484) [n:127.0.0.1:64301_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.testCreateShardRepFactor.b.replica1
[junit4] 2> 900760 INFO (qtp552936436-5489) [n:127.0.0.1:64302_solr ] o.a.s.c.SolrCore [testCreateShardRepFactor_a_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@595982c7
[junit4] 2> 900761 INFO (qtp1885666094-5484) [n:127.0.0.1:64301_solr ] o.a.s.c.SolrCore [testCreateShardRepFactor_b_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@39a32d23
[junit4] 2> 900761 INFO (qtp1474850440-5505) [n:127.0.0.1:64304_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.testCreateShardRepFactor.x.replica1
[junit4] 2> 900762 INFO (qtp1474850440-5505) [n:127.0.0.1:64304_solr ] o.a.s.c.SolrCore [testCreateShardRepFactor_x_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@306272d6
[junit4] 2> 900766 INFO (qtp1885666094-5484) [n:127.0.0.1:64301_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.testCreateShardRepFactor.b.replica1
[junit4] 2> 900766 INFO (qtp552936436-5489) [n:127.0.0.1:64302_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.testCreateShardRepFactor.a.replica1
[junit4] 2> 900767 INFO (qtp1474850440-5505) [n:127.0.0.1:64304_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.testCreateShardRepFactor.x.replica1
[junit4] 2> 900776 INFO (qtp552936436-5489) [n:127.0.0.1:64302_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=testCreateShardRepFactor_a_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=18
[junit4] 2> 900778 INFO (qtp1885666094-5484) [n:127.0.0.1:64301_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=testCreateShardRepFactor_b_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=20
[junit4] 2> 900799 INFO (qtp1474850440-5505) [n:127.0.0.1:64304_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=testCreateShardRepFactor_x_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=39
[junit4] 2> 901509 INFO (qtp1885666094-5483) [n:127.0.0.1:64301_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=testCreateShardRepFactor&action=DELETE&wt=javabin&version=2} status=0 QTime=758
[junit4] 2> 901510 INFO (qtp1885666094-5477) [n:127.0.0.1:64301_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params shards=a,b,c,d&replicationFactor=4&collection.configName=conf&maxShardsPerNode=5&router.field=shard_s&name=withShardField&router.name=implicit&action=CREATE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 901515 INFO (OverseerThreadFactory-2585-thread-4-processing-n:127.0.0.1:64303_solr) [n:127.0.0.1:64303_solr ] o.a.s.c.CreateCollectionCmd Create collection withShardField
[junit4] 2> 901515 INFO (OverseerCollectionConfigSetProcessor-97424739086958601-127.0.0.1:64303_solr-n_0000000000) [n:127.0.0.1:64303_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 901732 INFO (qtp1885666094-5481) [n:127.0.0.1:64301_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_d_replica2&action=CREATE&numShards=4&collection=withShardField&shard=d&wt=javabin&version=2
[junit4] 2> 901736 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_c_replica2&action=CREATE&numShards=4&collection=withShardField&shard=c&wt=javabin&version=2
[junit4] 2> 901738 INFO (qtp1885666094-5479) [n:127.0.0.1:64301_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_a_replica2&action=CREATE&numShards=4&collection=withShardField&shard=a&wt=javabin&version=2
[junit4] 2> 901748 INFO (qtp1885666094-5482) [n:127.0.0.1:64301_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_b_replica2&action=CREATE&numShards=4&collection=withShardField&shard=b&wt=javabin&version=2
[junit4] 2> 901751 INFO (qtp1474850440-5506) [n:127.0.0.1:64304_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_c_replica1&action=CREATE&numShards=4&collection=withShardField&shard=c&wt=javabin&version=2
[junit4] 2> 901754 INFO (qtp552936436-5490) [n:127.0.0.1:64302_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_a_replica3&action=CREATE&numShards=4&collection=withShardField&shard=a&wt=javabin&version=2
[junit4] 2> 901756 INFO (qtp1474850440-5507) [n:127.0.0.1:64304_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_a_replica1&action=CREATE&numShards=4&collection=withShardField&shard=a&wt=javabin&version=2
[junit4] 2> 901758 INFO (qtp928604155-5496) [n:127.0.0.1:64303_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_d_replica4&action=CREATE&numShards=4&collection=withShardField&shard=d&wt=javabin&version=2
[junit4] 2> 901758 INFO (qtp928604155-5500) [n:127.0.0.1:64303_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_a_replica4&action=CREATE&numShards=4&collection=withShardField&shard=a&wt=javabin&version=2
[junit4] 2> 901760 INFO (qtp928604155-5501) [n:127.0.0.1:64303_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_b_replica4&action=CREATE&numShards=4&collection=withShardField&shard=b&wt=javabin&version=2
[junit4] 2> 901760 INFO (qtp928604155-5502) [n:127.0.0.1:64303_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_c_replica4&action=CREATE&numShards=4&collection=withShardField&shard=c&wt=javabin&version=2
[junit4] 2> 901761 INFO (qtp552936436-5491) [n:127.0.0.1:64302_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_d_replica3&action=CREATE&numShards=4&collection=withShardField&shard=d&wt=javabin&version=2
[junit4] 2> 901761 INFO (qtp552936436-5492) [n:127.0.0.1:64302_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_c_replica3&action=CREATE&numShards=4&collection=withShardField&shard=c&wt=javabin&version=2
[junit4] 2> 901762 INFO (qtp1474850440-5509) [n:127.0.0.1:64304_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_b_replica1&action=CREATE&numShards=4&collection=withShardField&shard=b&wt=javabin&version=2
[junit4] 2> 901762 INFO (qtp552936436-5493) [n:127.0.0.1:64302_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_b_replica3&action=CREATE&numShards=4&collection=withShardField&shard=b&wt=javabin&version=2
[junit4] 2> 901763 INFO (qtp1474850440-5510) [n:127.0.0.1:64304_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=withShardField_d_replica1&action=CREATE&numShards=4&collection=withShardField&shard=d&wt=javabin&version=2
[junit4] 2> 901919 INFO (zkCallback-984-thread-2-processing-n:127.0.0.1:64303_solr) [n:127.0.0.1:64303_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/withShardField/state.json] for collection [withShardField] has occurred - updating... (live nodes size: [4])
[junit4] 2> 901919 INFO (zkCallback-985-thread-1-processing-n:127.0.0.1:64301_solr) [n:127.0.0.1:64301_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/withShardField/state.json] for collection [withShardField] has occurred - updating... (live nodes size: [4])
[junit4] 2> 901919 INFO (zkCallback-986-thread-1-processing-n:127.0.0.1:64302_solr) [n:127.0.0.1:64302_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/withShardField/state.json] for collection [withShardField] has occurred - updating... (live nodes size: [4])
[junit4] 2> 901919 INFO (zkCallback-987-thread-1-processing-n:127.0.0.1:64304_solr) [n:127.0.0.1:64304_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/withShardField/state.json] for collection [withShardField] has occurred - updating... (live nodes size: [4])
[junit4] 2> 902770 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:withShardField s:c x:withShardField_c_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 902770 INFO (qtp1885666094-5481) [n:127.0.0.1:64301_solr c:withShardField s:d x:withShardField_d_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 902770 INFO (qtp1885666094-5479) [n:127.0.0.1:64301_solr c:withShardField s:a x:withShardField_a_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 902811 INFO (qtp1885666094-5479) [n:127.0.0.1:64301_solr c:withShardField s:a x:withShardField_a_replica2] o.a.s.s.IndexSchema [withShardField_a_replica2] Schema name=cloud-dynamic
[junit4] 2> 902812 INFO (qtp1885666094-5481) [n:127.0.0.1:64301_solr c:withShardField s:d x:withShardField_d_replica2] o.a.s.s.IndexSchema [withShardField_d_replica2] Schema name=cloud-dynamic
[junit4] 2> 902833 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:withShardField s:c x:withShardField_c_replica2] o.a.s.s.IndexSchema [withShardField_c_replica2] Schema name=cloud-dynamic
[junit4] 2> 902863 WARN (qtp1885666094-5479) [n:127.0.0.1:64301_solr c:withShardField s:a x:withShardField_a_replica2] o.a.s.s.IndexSchema [withShardField_a_replica2] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 902864 INFO (qtp1885666094-5479) [n:127.0.0.1:64301_solr c:withShardField s:a x:withShardField_a_replica2] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 902868 WARN (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:withShardField s:c x:withShardField_c_replica2] o.a.s.s.IndexSchema [withShardField_c_replica2] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 902869 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:withShardField s:c x:withShardField_c_replica2] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 902873 WARN (qtp1885666094-5481) [n:127.0.0.1:64301_solr c:withShardField s:d x:withShardField_d_replica2] o.a.s.s.IndexSchema [withShardField_d_replica2] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 902874 INFO (qtp1885666094-5481) [n:127.0.0.1:64301_solr c:withShardField s:d x:withShardField_d_replica2] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 902876 INFO (qtp1885666094-5479) [n:127.0.0.1:64301_solr c:withShardField s:a x:withShardField_a_replica2] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_a_replica2' using configuration from collection withShardField
[junit4] 2> 902876 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:withShardField s:c x:withShardField_c_replica2] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_c_replica2' using configuration from collection withShardField
[junit4] 2> 902876 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:withShardField s:c x:withShardField_c_replica2] o.a.s.c.SolrCore [[withShardField_c_replica2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node4/withShardField_c_replica2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node4/./withShardField_c_replica2/data/]
[junit4] 2> 902879 INFO (qtp1885666094-5479) [n:127.0.0.1:64301_solr c:withShardField s:a x:withShardField_a_replica2] o.a.s.c.SolrCore [[withShardField_a_replica2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node4/withShardField_a_replica2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node4/./withShardField_a_replica2/data/]
[junit4] 2> 902894 INFO (qtp1885666094-5481) [n:127.0.0.1:64301_solr c:withShardField s:d x:withShardField_d_replica2] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_d_replica2' using configuration from collection withShardField
[junit4] 2> 902896 INFO (qtp1885666094-5482) [n:127.0.0.1:64301_solr c:withShardField s:b x:withShardField_b_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 902896 INFO (qtp1885666094-5481) [n:127.0.0.1:64301_solr c:withShardField s:d x:withShardField_d_replica2] o.a.s.c.SolrCore [[withShardField_d_replica2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node4/withShardField_d_replica2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node4/./withShardField_d_replica2/data/]
[junit4] 2> 902901 INFO (qtp552936436-5493) [n:127.0.0.1:64302_solr c:withShardField s:b x:withShardField_b_replica3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 902904 INFO (qtp928604155-5502) [n:127.0.0.1:64303_solr c:withShardField s:c x:withShardField_c_replica4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 902923 INFO (qtp552936436-5490) [n:127.0.0.1:64302_solr c:withShardField s:a x:withShardField_a_replica3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 902925 INFO (qtp1474850440-5510) [n:127.0.0.1:64304_solr c:withShardField s:d x:withShardField_d_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 902927 INFO (qtp1474850440-5507) [n:127.0.0.1:64304_solr c:withShardField s:a x:withShardField_a_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 902941 INFO (qtp552936436-5491) [n:127.0.0.1:64302_solr c:withShardField s:d x:withShardField_d_replica3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 902948 INFO (qtp552936436-5492) [n:127.0.0.1:64302_solr c:withShardField s:c x:withShardField_c_replica3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 902965 INFO (qtp928604155-5501) [n:127.0.0.1:64303_solr c:withShardField s:b x:withShardField_b_replica4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 902968 INFO (qtp928604155-5500) [n:127.0.0.1:64303_solr c:withShardField s:a x:withShardField_a_replica4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 902986 INFO (qtp1474850440-5506) [n:127.0.0.1:64304_solr c:withShardField s:c x:withShardField_c_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 902993 INFO (qtp1474850440-5509) [n:127.0.0.1:64304_solr c:withShardField s:b x:withShardField_b_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 902995 INFO (qtp928604155-5496) [n:127.0.0.1:64303_solr c:withShardField s:d x:withShardField_d_replica4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 903010 INFO (qtp1474850440-5507) [n:127.0.0.1:64304_solr c:withShardField s:a x:withShardField_a_replica1] o.a.s.s.IndexSchema [withShardField_a_replica1] Schema name=cloud-dynamic
[junit4] 2> 903012 INFO (qtp552936436-5493) [n:127.0.0.1:64302_solr c:withShardField s:b x:withShardField_b_replica3] o.a.s.s.IndexSchema [withShardField_b_replica3] Schema name=cloud-dynamic
[junit4] 2> 903019 INFO (qtp552936436-5490) [n:127.0.0.1:64302_solr c:withShardField s:a x:withShardField_a_replica3] o.a.s.s.IndexSchema [withShardField_a_replica3] Schema name=cloud-dynamic
[junit4] 2> 903038 INFO (qtp928604155-5502) [n:127.0.0.1:64303_solr c:withShardField s:c x:withShardField_c_replica4] o.a.s.s.IndexSchema [withShardField_c_replica4] Schema name=cloud-dynamic
[junit4] 2> 903038 INFO (qtp552936436-5491) [n:127.0.0.1:64302_solr c:withShardField s:d x:withShardField_d_replica3] o.a.s.s.IndexSchema [withShardField_d_replica3] Schema name=cloud-dynamic
[junit4] 2> 903041 INFO (qtp552936436-5492) [n:127.0.0.1:64302_solr c:withShardField s:c x:withShardField_c_replica3] o.a.s.s.IndexSchema [withShardField_c_replica3] Schema name=cloud-dynamic
[junit4] 2> 903041 INFO (qtp928604155-5500) [n:127.0.0.1:64303_solr c:withShardField s:a x:withShardField_a_replica4] o.a.s.s.IndexSchema [withShardField_a_replica4] Schema name=cloud-dynamic
[junit4] 2> 903041 INFO (qtp928604155-5501) [n:127.0.0.1:64303_solr c:withShardField s:b x:withShardField_b_replica4] o.a.s.s.IndexSchema [withShardField_b_replica4] Schema name=cloud-dynamic
[junit4] 2> 903045 INFO (qtp1474850440-5506) [n:127.0.0.1:64304_solr c:withShardField s:c x:withShardField_c_replica1] o.a.s.s.IndexSchema [withShardField_c_replica1] Schema name=cloud-dynamic
[junit4] 2> 903046 INFO (qtp1474850440-5510) [n:127.0.0.1:64304_solr c:withShardField s:d x:withShardField_d_replica1] o.a.s.s.IndexSchema [withShardField_d_replica1] Schema name=cloud-dynamic
[junit4] 2> 903055 INFO (qtp1885666094-5482) [n:127.0.0.1:64301_solr c:withShardField s:b x:withShardField_b_replica2] o.a.s.s.IndexSchema [withShardField_b_replica2] Schema name=cloud-dynamic
[junit4] 2> 903108 INFO (qtp1474850440-5509) [n:127.0.0.1:64304_solr c:withShardField s:b x:withShardField_b_replica1] o.a.s.s.IndexSchema [withShardField_b_replica1] Schema name=cloud-dynamic
[junit4] 2> 903141 INFO (qtp928604155-5496) [n:127.0.0.1:64303_solr c:withShardField s:d x:withShardField_d_replica4] o.a.s.s.IndexSchema [withShardField_d_replica4] Schema name=cloud-dynamic
[junit4] 2> 903176 WARN (qtp1885666094-5482) [n:127.0.0.1:64301_solr c:withShardField s:b x:withShardField_b_replica2] o.a.s.s.IndexSchema [withShardField_b_replica2] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 903177 INFO (qtp1885666094-5482) [n:127.0.0.1:64301_solr c:withShardField s:b x:withShardField_b_replica2] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 903180 WARN (qtp1474850440-5506) [n:127.0.0.1:64304_solr c:withShardField s:c x:withShardField_c_replica1] o.a.s.s.IndexSchema [withShardField_c_replica1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 903182 INFO (qtp1474850440-5506) [n:127.0.0.1:64304_solr c:withShardField s:c x:withShardField_c_replica1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 903183 INFO (qtp1885666094-5482) [n:127.0.0.1:64301_solr c:withShardField s:b x:withShardField_b_replica2] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_b_replica2' using configuration from collection withShardField
[junit4] 2> 903184 INFO (qtp1885666094-5482) [n:127.0.0.1:64301_solr c:withShardField s:b x:withShardField_b_replica2] o.a.s.c.SolrCore [[withShardField_b_replica2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node4/withShardField_b_replica2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node4/./withShardField_b_replica2/data/]
[junit4] 2> 903268 INFO (qtp1474850440-5506) [n:127.0.0.1:64304_solr c:withShardField s:c x:withShardField_c_replica1] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_c_replica1' using configuration from collection withShardField
[junit4] 2> 903269 INFO (qtp1474850440-5506) [n:127.0.0.1:64304_solr c:withShardField s:c x:withShardField_c_replica1] o.a.s.c.SolrCore [[withShardField_c_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node1/withShardField_c_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node1/./withShardField_c_replica1/data/]
[junit4] 2> 903277 WARN (qtp928604155-5496) [n:127.0.0.1:64303_solr c:withShardField s:d x:withShardField_d_replica4] o.a.s.s.IndexSchema [withShardField_d_replica4] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 903278 INFO (qtp928604155-5496) [n:127.0.0.1:64303_solr c:withShardField s:d x:withShardField_d_replica4] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 903278 WARN (qtp928604155-5501) [n:127.0.0.1:64303_solr c:withShardField s:b x:withShardField_b_replica4] o.a.s.s.IndexSchema [withShardField_b_replica4] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 903279 INFO (qtp928604155-5501) [n:127.0.0.1:64303_solr c:withShardField s:b x:withShardField_b_replica4] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 903279 WARN (qtp552936436-5490) [n:127.0.0.1:64302_solr c:withShardField s:a x:withShardField_a_replica3] o.a.s.s.IndexSchema [withShardField_a_replica3] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 903280 INFO (qtp552936436-5490) [n:127.0.0.1:64302_solr c:withShardField s:a x:withShardField_a_replica3] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 903280 WARN (qtp1474850440-5510) [n:127.0.0.1:64304_solr c:withShardField s:d x:withShardField_d_replica1] o.a.s.s.IndexSchema [withShardField_d_replica1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 903280 INFO (qtp1474850440-5510) [n:127.0.0.1:64304_solr c:withShardField s:d x:withShardField_d_replica1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 903281 WARN (qtp552936436-5493) [n:127.0.0.1:64302_solr c:withShardField s:b x:withShardField_b_replica3] o.a.s.s.IndexSchema [withShardField_b_replica3] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 903281 INFO (qtp552936436-5493) [n:127.0.0.1:64302_solr c:withShardField s:b x:withShardField_b_replica3] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 903282 INFO (qtp1474850440-5510) [n:127.0.0.1:64304_solr c:withShardField s:d x:withShardField_d_replica1] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_d_replica1' using configuration from collection withShardField
[junit4] 2> 903282 INFO (qtp1474850440-5510) [n:127.0.0.1:64304_solr c:withShardField s:d x:withShardField_d_replica1] o.a.s.c.SolrCore [[withShardField_d_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node1/withShardField_d_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node1/./withShardField_d_replica1/data/]
[junit4] 2> 903283 INFO (qtp552936436-5493) [n:127.0.0.1:64302_solr c:withShardField s:b x:withShardField_b_replica3] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_b_replica3' using configuration from collection withShardField
[junit4] 2> 903283 INFO (qtp552936436-5493) [n:127.0.0.1:64302_solr c:withShardField s:b x:withShardField_b_replica3] o.a.s.c.SolrCore [[withShardField_b_replica3] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node2/withShardField_b_replica3], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node2/./withShardField_b_replica3/data/]
[junit4] 2> 903284 INFO (qtp552936436-5490) [n:127.0.0.1:64302_solr c:withShardField s:a x:withShardField_a_replica3] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_a_replica3' using configuration from collection withShardField
[junit4] 2> 903284 INFO (qtp552936436-5490) [n:127.0.0.1:64302_solr c:withShardField s:a x:withShardField_a_replica3] o.a.s.c.SolrCore [[withShardField_a_replica3] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node2/withShardField_a_replica3], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node2/./withShardField_a_replica3/data/]
[junit4] 2> 903286 INFO (qtp928604155-5501) [n:127.0.0.1:64303_solr c:withShardField s:b x:withShardField_b_replica4] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_b_replica4' using configuration from collection withShardField
[junit4] 2> 903286 INFO (qtp928604155-5501) [n:127.0.0.1:64303_solr c:withShardField s:b x:withShardField_b_replica4] o.a.s.c.SolrCore [[withShardField_b_replica4] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node3/withShardField_b_replica4], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node3/./withShardField_b_replica4/data/]
[junit4] 2> 903288 INFO (qtp928604155-5496) [n:127.0.0.1:64303_solr c:withShardField s:d x:withShardField_d_replica4] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_d_replica4' using configuration from collection withShardField
[junit4] 2> 903288 INFO (qtp928604155-5496) [n:127.0.0.1:64303_solr c:withShardField s:d x:withShardField_d_replica4] o.a.s.c.SolrCore [[withShardField_d_replica4] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node3/withShardField_d_replica4], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node3/./withShardField_d_replica4/data/]
[junit4] 2> 903303 WARN (qtp1474850440-5507) [n:127.0.0.1:64304_solr c:withShardField s:a x:withShardField_a_replica1] o.a.s.s.IndexSchema [withShardField_a_replica1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 903303 INFO (qtp1474850440-5507) [n:127.0.0.1:64304_solr c:withShardField s:a x:withShardField_a_replica1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 903305 INFO (qtp1474850440-5507) [n:127.0.0.1:64304_solr c:withShardField s:a x:withShardField_a_replica1] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_a_replica1' using configuration from collection withShardField
[junit4] 2> 903308 WARN (qtp928604155-5502) [n:127.0.0.1:64303_solr c:withShardField s:c x:withShardField_c_replica4] o.a.s.s.IndexSchema [withShardField_c_replica4] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 903309 INFO (qtp1474850440-5507) [n:127.0.0.1:64304_solr c:withShardField s:a x:withShardField_a_replica1] o.a.s.c.SolrCore [[withShardField_a_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node1/withShardField_a_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node1/./withShardField_a_replica1/data/]
[junit4] 2> 903310 INFO (qtp928604155-5502) [n:127.0.0.1:64303_solr c:withShardField s:c x:withShardField_c_replica4] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 903312 INFO (qtp928604155-5502) [n:127.0.0.1:64303_solr c:withShardField s:c x:withShardField_c_replica4] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_c_replica4' using configuration from collection withShardField
[junit4] 2> 903313 WARN (qtp552936436-5492) [n:127.0.0.1:64302_solr c:withShardField s:c x:withShardField_c_replica3] o.a.s.s.IndexSchema [withShardField_c_replica3] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 903314 INFO (qtp552936436-5492) [n:127.0.0.1:64302_solr c:withShardField s:c x:withShardField_c_replica3] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 903315 INFO (qtp552936436-5492) [n:127.0.0.1:64302_solr c:withShardField s:c x:withShardField_c_replica3] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_c_replica3' using configuration from collection withShardField
[junit4] 2> 903316 INFO (qtp552936436-5492) [n:127.0.0.1:64302_solr c:withShardField s:c x:withShardField_c_replica3] o.a.s.c.SolrCore [[withShardField_c_replica3] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node2/withShardField_c_replica3], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node2/./withShardField_c_replica3/data/]
[junit4] 2> 903317 INFO (qtp928604155-5502) [n:127.0.0.1:64303_solr c:withShardField s:c x:withShardField_c_replica4] o.a.s.c.SolrCore [[withShardField_c_replica4] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node3/withShardField_c_replica4], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node3/./withShardField_c_replica4/data/]
[junit4] 2> 903320 WARN (qtp928604155-5500) [n:127.0.0.1:64303_solr c:withShardField s:a x:withShardField_a_replica4] o.a.s.s.IndexSchema [withShardField_a_replica4] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 903321 INFO (qtp928604155-5500) [n:127.0.0.1:64303_solr c:withShardField s:a x:withShardField_a_replica4] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 903323 INFO (qtp928604155-5500) [n:127.0.0.1:64303_solr c:withShardField s:a x:withShardField_a_replica4] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_a_replica4' using configuration from collection withShardField
[junit4] 2> 903324 INFO (qtp928604155-5500) [n:127.0.0.1:64303_solr c:withShardField s:a x:withShardField_a_replica4] o.a.s.c.SolrCore [[withShardField_a_replica4] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node3/withShardField_a_replica4], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node3/./withShardField_a_replica4/data/]
[junit4] 2> 903329 WARN (qtp552936436-5491) [n:127.0.0.1:64302_solr c:withShardField s:d x:withShardField_d_replica3] o.a.s.s.IndexSchema [withShardField_d_replica3] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 903330 INFO (qtp552936436-5491) [n:127.0.0.1:64302_solr c:withShardField s:d x:withShardField_d_replica3] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 903331 INFO (qtp552936436-5491) [n:127.0.0.1:64302_solr c:withShardField s:d x:withShardField_d_replica3] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_d_replica3' using configuration from collection withShardField
[junit4] 2> 903332 INFO (qtp552936436-5491) [n:127.0.0.1:64302_solr c:withShardField s:d x:withShardField_d_replica3] o.a.s.c.SolrCore [[withShardField_d_replica3] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node2/withShardField_d_replica3], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node2/./withShardField_d_replica3/data/]
[junit4] 2> 903353 WARN (qtp1474850440-5509) [n:127.0.0.1:64304_solr c:withShardField s:b x:withShardField_b_replica1] o.a.s.s.IndexSchema [withShardField_b_replica1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 903354 INFO (qtp1474850440-5509) [n:127.0.0.1:64304_solr c:withShardField s:b x:withShardField_b_replica1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 903355 INFO (qtp1474850440-5509) [n:127.0.0.1:64304_solr c:withShardField s:b x:withShardField_b_replica1] o.a.s.c.CoreContainer Creating SolrCore 'withShardField_b_replica1' using configuration from collection withShardField
[junit4] 2> 903355 INFO (qtp1474850440-5509) [n:127.0.0.1:64304_solr c:withShardField s:b x:withShardField_b_replica1] o.a.s.c.SolrCore [[withShardField_b_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node1/withShardField_b_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_D62FC493DF153A9E-001/tempDir-001/node1/./withShardField_b_replica1/data/]
[junit4] 2> 903368 INFO (qtp1885666094-5481) [n:127.0.0.1:64301_solr c:withShardField s:d x:withShardField_d_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 903368 INFO (qtp1885666094-5481) [n:127.0.0.1:64301_solr c:withShardField s:d x:withShardField_d_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 903376 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:withShardField s:c x:withShardField_c_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 903376 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:withShardField s:c x:withShardField_c_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 903378 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:withShardField s:c x:withShardField_c_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 903378 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr c:withShardField s:c x:withShardField_c_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 903381 INFO (qtp1885666094-5481) [n:127.0.0.1:64301_solr c:withShardField s:d x:withShardField_d_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 903381 INFO (qtp1885666094-5481) [n:127.0.0.1:64301_solr c:withShardField s:d x:withShardField_d_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 903430 INFO (qtp1885666094-5480) [n:127.0.0.1:64301_solr
[...truncated too long message...]
tdown
[junit4] 2> 1204615 INFO (zkCallback-1111-thread-5-processing-n:127.0.0.1:64594_) [n:127.0.0.1:64594_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1208650 INFO (recoveryExecutor-1109-thread-1-processing-n:127.0.0.1:64594_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:64594_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy RecoveryStrategy has been closed
[junit4] 2> 1208650 INFO (recoveryExecutor-1109-thread-1-processing-n:127.0.0.1:64594_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:64594_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Finished recovery process, successful=[false]
[junit4] 2> 1208650 INFO (TEST-PeerSyncReplicationTest.test-seed#[D62FC493DF153A9E]) [ ] o.a.s.c.Overseer Overseer (id=97424746072178706-127.0.0.1:64594_-n_0000000004) closing
[junit4] 2> 1208650 INFO (OverseerStateUpdate-97424746072178706-127.0.0.1:64594_-n_0000000004) [n:127.0.0.1:64594_ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:64594_
[junit4] 2> 1208652 WARN (zkCallback-1111-thread-5-processing-n:127.0.0.1:64594_) [n:127.0.0.1:64594_ ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 1210161 INFO (zkCallback-1111-thread-3-processing-n:127.0.0.1:64594_) [n:127.0.0.1:64594_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@36cb4625
[junit4] 2> 1210162 WARN (zkCallback-1111-thread-3-processing-n:127.0.0.1:64594_) [n:127.0.0.1:64594_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Stopping recovery for core=[collection1] coreNodeName=[core_node2]
[junit4] 2> 1210205 INFO (zkCallback-1111-thread-3-processing-n:127.0.0.1:64594_) [n:127.0.0.1:64594_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection1
[junit4] 2> 1210206 INFO (TEST-PeerSyncReplicationTest.test-seed#[D62FC493DF153A9E]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 1210207 INFO (TEST-PeerSyncReplicationTest.test-seed#[D62FC493DF153A9E]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@149f2481{HTTP/1.1,[http/1.1]}{127.0.0.1:64594}
[junit4] 2> 1210207 INFO (TEST-PeerSyncReplicationTest.test-seed#[D62FC493DF153A9E]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6a4ec715{/,null,UNAVAILABLE}
[junit4] 2> 1210208 INFO (TEST-PeerSyncReplicationTest.test-seed#[D62FC493DF153A9E]) [ ] o.a.s.c.ChaosMonkey monkey: stop shard! 64601
[junit4] 2> 1210210 INFO (TEST-PeerSyncReplicationTest.test-seed#[D62FC493DF153A9E]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:64578 64578
[junit4] 2> 1210262 INFO (Thread-1553) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:64578 64578
[junit4] 2> 1213215 WARN (Thread-1553) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/aliases.json
[junit4] 2> 6 /solr/clusterprops.json
[junit4] 2> 5 /solr/security.json
[junit4] 2> 5 /solr/configs/conf1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/clusterstate.json
[junit4] 2> 4 /solr/collections/collection1/state.json
[junit4] 2> 2 /solr/overseer_elect/election/97424746072178697-127.0.0.1:64588_-n_0000000001
[junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/97424746072178697-core_node1-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 207 /solr/overseer/collection-queue-work
[junit4] 2> 41 /solr/overseer/queue
[junit4] 2> 15 /solr/overseer/queue-work
[junit4] 2> 6 /solr/collections
[junit4] 2> 5 /solr/live_nodes
[junit4] 2>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=PeerSyncReplicationTest -Dtests.method=test -Dtests.seed=D62FC493DF153A9E -Dtests.slow=true -Dtests.locale=zh -Dtests.timezone=Israel -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 212s J1 | PeerSyncReplicationTest.test <<<
[junit4] > Throwable #1: java.lang.AssertionError: timeout waiting to see all nodes active
[junit4] > at __randomizedtesting.SeedInfo.seed([D62FC493DF153A9E:5E7BFB4971E95766]:0)
[junit4] > at org.apache.solr.cloud.PeerSyncReplicationTest.waitTillNodesActive(PeerSyncReplicationTest.java:326)
[junit4] > at org.apache.solr.cloud.PeerSyncReplicationTest.bringUpDeadNodeAndEnsureNoReplication(PeerSyncReplicationTest.java:277)
[junit4] > at org.apache.solr.cloud.PeerSyncReplicationTest.forceNodeFailureAndDoPeerSync(PeerSyncReplicationTest.java:259)
[junit4] > at org.apache.solr.cloud.PeerSyncReplicationTest.test(PeerSyncReplicationTest.java:138)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 1213217 INFO (SUITE-PeerSyncReplicationTest-seed#[D62FC493DF153A9E]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_D62FC493DF153A9E-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {other_tl1=FSTOrd50, range_facet_l_dv=PostingsFormat(name=Asserting), rnd_s=Lucene50(blocksize=128), multiDefault=Lucene50(blocksize=128), a_t=FSTOrd50, intDefault=FSTOrd50, a_i1=FSTOrd50, id=PostingsFormat(name=Asserting), range_facet_i_dv=FSTOrd50, text=PostingsFormat(name=MockRandom), range_facet_l=FSTOrd50, timestamp=FSTOrd50}, docValues:{range_facet_l_dv=DocValuesFormat(name=Lucene70), _version_=DocValuesFormat(name=Direct), range_facet_i_dv=DocValuesFormat(name=Direct), intDvoDefault=DocValuesFormat(name=Memory), timestamp=DocValuesFormat(name=Direct)}, maxPointsInLeafNode=654, maxMBSortInHeap=6.3893751817683055, sim=RandomSimilarity(queryNorm=true): {}, locale=zh, timezone=Israel
[junit4] 2> NOTE: Mac OS X 10.11.6 x86_64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=3,threads=1,free=60485480,total=372834304
[junit4] 2> NOTE: All tests run in this JVM: [DistanceUnitsTest, CoreMergeIndexesAdminHandlerTest, TestSolrQueryParser, TestBinaryResponseWriter, ChaosMonkeySafeLeaderTest, ShardRoutingCustomTest, NoCacheHeaderTest, TestExportWriter, SolrIndexSplitterTest, SpellCheckComponentTest, TestPHPSerializedResponseWriter, TestExactSharedStatsCache, TemplateUpdateProcessorTest, LoggingHandlerTest, VMParamsZkACLAndCredentialsProvidersTest, BasicDistributedZk2Test, LeaderElectionIntegrationTest, TestCustomStream, TestOverriddenPrefixQueryForCustomFieldType, TestSolrCloudWithDelegationTokens, TestCustomSort, AutoCommitTest, SuggesterTSTTest, TestSolrIndexConfig, PreAnalyzedFieldTest, TestDocBasedVersionConstraints, TestHdfsBackupRestoreCore, TestAnalyzeInfixSuggestions, CdcrUpdateLogTest, TestShortCircuitedRequests, TestConfigSetImmutable, BlockDirectoryTest, RollingRestartTest, DistributedIntervalFacetingTest, DistributedQueryComponentCustomSortTest, TestSolrQueryParserDefaultOperatorResource, FieldAnalysisRequestHandlerTest, HLLSerializationTest, TestFileDictionaryLookup, HighlighterConfigTest, DistributedFacetPivotSmallTest, TestSuggestSpellingConverter, UpdateRequestProcessorFactoryTest, TestJmxMonitoredMap, TestManagedResource, TestXmlQParser, SpellCheckCollatorTest, TestDFISimilarityFactory, SolrCmdDistributorTest, TestDynamicFieldResource, TestMacroExpander, EchoParamsTest, UtilsToolTest, ConcurrentDeleteAndCreateCollectionTest, StatsComponentTest, OverriddenZkACLAndCredentialsProvidersTest, TestFieldResource, TestBulkSchemaConcurrent, TestConfig, DateRangeFieldTest, ResponseHeaderTest, TestConfigsApi, RemoteQueryErrorTest, TestPushWriter, HdfsDirectoryTest, AnalyticsQueryTest, TestAnalyzedSuggestions, AssignTest, TermVectorComponentDistributedTest, TlogReplayBufferedWhileIndexingTest, TestRealTimeGet, IgnoreCommitOptimizeUpdateProcessorFactoryTest, EnumFieldTest, TestSchemaVersionResource, TestRuleBasedAuthorizationPlugin, JsonLoaderTest, SimpleCollectionCreateDeleteTest, CurrencyFieldOpenExchangeTest, ConnectionManagerTest, HdfsNNFailoverTest, DistributedQueueTest, MinimalSchemaTest, TestFieldCacheReopen, ResponseLogComponentTest, TestStressInPlaceUpdates, TestMinMaxOnMultiValuedField, TestSearchPerf, TestPerFieldSimilarityWithDefaultOverride, UUIDFieldTest, SortSpecParsingTest, TestCloudPseudoReturnFields, TestAtomicUpdateErrorCases, DocValuesNotIndexedTest, TestReqParamsAPI, TestDFRSimilarityFactory, ClassificationUpdateProcessorIntegrationTest, TestInitParams, TestDelegationWithHadoopAuth, TermVectorComponentTest, TestSolr4Spatial, TestHdfsCloudBackupRestore, TestLocalFSCloudBackupRestore, CustomCollectionTest, TestSimpleTrackingShardHandler, TestTrie, TestObjectReleaseTracker, CircularListTest, TestPathTrie, TestClusterProperties, TestFieldCacheVsDocValues, SignatureUpdateProcessorFactoryTest, ShufflingReplicaListTransformerTest, HdfsRecoveryZkTest, TestLRUStatsCache, JSONWriterTest, PeerSyncReplicationTest]
[junit4] Completed [221/693 (2!)] on J1 in 212.00s, 1 test, 1 failure <<< FAILURES!
[...truncated 63972 lines...]
[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3819
- Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3819/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC
6 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithDelegationTokens
Error Message:
java.io.IOException: Couldn't instantiate org.apache.zookeeper.ClientCnxnSocketNIO
Stack Trace:
org.apache.solr.common.SolrException: java.io.IOException: Couldn't instantiate org.apache.zookeeper.ClientCnxnSocketNIO
at __randomizedtesting.SeedInfo.seed([357F96641734EF54]:0)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:170)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:116)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:111)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:98)
at org.apache.solr.cloud.MiniSolrCloudCluster.waitForAllNodes(MiniSolrCloudCluster.java:260)
at org.apache.solr.cloud.MiniSolrCloudCluster.<init>(MiniSolrCloudCluster.java:254)
at org.apache.solr.cloud.MiniSolrCloudCluster.<init>(MiniSolrCloudCluster.java:187)
at org.apache.solr.cloud.MiniSolrCloudCluster.<init>(MiniSolrCloudCluster.java:114)
at org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.startup(TestSolrCloudWithDelegationTokens.java:69)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:847)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Couldn't instantiate org.apache.zookeeper.ClientCnxnSocketNIO
at org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:1782)
at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:447)
at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:380)
at org.apache.solr.common.cloud.SolrZooKeeper.<init>(SolrZooKeeper.java:43)
at org.apache.solr.common.cloud.ZkClientConnectionStrategy.createSolrZooKeeper(ZkClientConnectionStrategy.java:105)
at org.apache.solr.common.cloud.DefaultConnectionStrategy.connect(DefaultConnectionStrategy.java:37)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:145)
... 32 more
Caused by: java.io.IOException: Too many open files
at sun.nio.ch.IOUtil.makePipe(Native Method)
at sun.nio.ch.KQueueSelectorImpl.<init>(KQueueSelectorImpl.java:84)
at sun.nio.ch.KQueueSelectorProvider.openSelector(KQueueSelectorProvider.java:42)
at java.nio.channels.Selector.open(Selector.java:227)
at org.apache.zookeeper.ClientCnxnSocketNIO.<init>(ClientCnxnSocketNIO.java:43)
at sun.reflect.GeneratedConstructorAccessor227.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:1779)
... 38 more
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithDelegationTokens
Error Message:
Stack Trace:
java.lang.NullPointerException
at __randomizedtesting.SeedInfo.seed([357F96641734EF54]:0)
at org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.shutdown(TestSolrCloudWithDelegationTokens.java:86)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithDelegationTokens
Error Message:
ObjectTracker found 6 object(s) that were not released!!! [InternalHttpClient, InternalHttpClient, InternalHttpClient, SolrZkClient, InternalHttpClient, SolrZkClient] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:287) at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:195) at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:47) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:484) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:287) at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:90) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:490) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:287) at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:90) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:490) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:184) at org.apache.solr.cloud.ZkController.<init>(ZkController.java:296) at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:112) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:499) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:287) at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:195) at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:47) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:484) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:184) at org.apache.solr.cloud.ZkController.<init>(ZkController.java:296) at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:112) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:499) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
java.lang.AssertionError: ObjectTracker found 6 object(s) that were not released!!! [InternalHttpClient, InternalHttpClient, InternalHttpClient, SolrZkClient, InternalHttpClient, SolrZkClient]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:287)
at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:195)
at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:47)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:484)
at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169)
at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376)
at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:287)
at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:90)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:490)
at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169)
at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376)
at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:287)
at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:90)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:490)
at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169)
at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376)
at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:184)
at org.apache.solr.cloud.ZkController.<init>(ZkController.java:296)
at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:112)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:499)
at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169)
at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376)
at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:287)
at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:195)
at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:47)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:484)
at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169)
at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376)
at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:184)
at org.apache.solr.cloud.ZkController.<init>(ZkController.java:296)
at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:112)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:499)
at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169)
at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376)
at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([357F96641734EF54]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:289)
at sun.reflect.GeneratedMethodAccessor42.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithDelegationTokens
Error Message:
Could not remove the following files (in the order of attempts): /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithDelegationTokens_357F96641734EF54-001/tempDir-001: java.nio.file.FileSystemException: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithDelegationTokens_357F96641734EF54-001/tempDir-001: Too many open files /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithDelegationTokens_357F96641734EF54-001/init-core-data-001: java.nio.file.FileSystemException: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithDelegationTokens_357F96641734EF54-001/init-core-data-001: Too many open files /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithDelegationTokens_357F96641734EF54-001: java.nio.file.FileSystemException: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithDelegationTokens_357F96641734EF54-001: Too many open files
Stack Trace:
java.io.IOException: Could not remove the following files (in the order of attempts):
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithDelegationTokens_357F96641734EF54-001/tempDir-001: java.nio.file.FileSystemException: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithDelegationTokens_357F96641734EF54-001/tempDir-001: Too many open files
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithDelegationTokens_357F96641734EF54-001/init-core-data-001: java.nio.file.FileSystemException: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithDelegationTokens_357F96641734EF54-001/init-core-data-001: Too many open files
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithDelegationTokens_357F96641734EF54-001: java.nio.file.FileSystemException: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithDelegationTokens_357F96641734EF54-001: Too many open files
at org.apache.lucene.util.IOUtils.rm(IOUtils.java:323)
at org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:216)
at com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithDelegationTokens
Error Message:
55 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithDelegationTokens: 1) Thread[id=19013, name=qtp344752888-19013, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 2) Thread[id=19025, name=qtp2110168235-19025, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 3) Thread[id=19018, name=qtp2110168235-19018, state=RUNNABLE, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:745) 4) Thread[id=19043, name=jetty-launcher-3412-thread-2-SendThread(127.0.0.1:64286), state=RUNNABLE, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) 5) Thread[id=19054, name=jetty-launcher-3412-thread-2-SendThread(127.0.0.1:64286), state=RUNNABLE, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) 6) Thread[id=19051, name=OverseerHdfsCoreFailoverThread-97421845135032325-127.0.0.1:64288_solr-n_0000000000, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:139) at java.lang.Thread.run(Thread.java:745) 7) Thread[id=19019, name=qtp2110168235-19019-acceptor-0@2e5b7fe0-ServerConnector@4b3e80d0{SSL,[ssl, http/1.1]}{127.0.0.1:64289}, state=RUNNABLE, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:373) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:745) 8) Thread[id=19035, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:745) 9) Thread[id=19000, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:178) at java.lang.Thread.run(Thread.java:745) 10) Thread[id=19071, name=pool-11-thread-1, state=WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 11) Thread[id=19047, name=zkCallback-3422-thread-1, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 12) Thread[id=19060, name=pool-8-thread-1, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 13) Thread[id=19050, name=OverseerCollectionConfigSetProcessor-97421845135032325-127.0.0.1:64288_solr-n_0000000000, state=TIMED_WAITING, group=Overseer collection creation process.] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.apache.solr.cloud.DistributedQueue.peekElements(DistributedQueue.java:343) at org.apache.solr.cloud.OverseerTaskQueue.peekTopN(OverseerTaskQueue.java:235) at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:227) at java.lang.Thread.run(Thread.java:745) 14) Thread[id=19020, name=qtp2110168235-19020, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 15) Thread[id=19001, name=SessionTracker, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at java.lang.Object.wait(Native Method) at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146) 16) Thread[id=19062, name=Curator-PathChildrenCache-0, state=WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 17) Thread[id=19036, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:745) 18) Thread[id=19069, name=Thread[Thread-4404,5,TGRP-TestSolrCloudWithDelegationTokens], state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at java.lang.Thread.sleep(Native Method) at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager$ExpiredTokenRemover.run(AbstractDelegationTokenSecretManager.java:657) at java.lang.Thread.run(Thread.java:745) 19) Thread[id=19057, name=jetty-launcher-3412-thread-1-EventThread, state=WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494) 20) Thread[id=19061, name=pool-9-thread-1, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 21) Thread[id=19023, name=qtp2110168235-19023, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 22) Thread[id=19015, name=qtp344752888-19015, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 23) Thread[id=19026, name=org.eclipse.jetty.server.session.HashSessionManager@174f5552Timer, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 24) Thread[id=19059, name=Curator-Framework-0, state=WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.DelayQueue.take(DelayQueue.java:211) at java.util.concurrent.DelayQueue.take(DelayQueue.java:70) at org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:780) at org.apache.curator.framework.imps.CuratorFrameworkImpl.access$300(CuratorFrameworkImpl.java:62) at org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call(CuratorFrameworkImpl.java:257) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 25) Thread[id=19052, name=Curator-ConnectionStateManager-0, state=WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:403) at org.apache.curator.framework.state.ConnectionStateManager.processEvents(ConnectionStateManager.java:245) at org.apache.curator.framework.state.ConnectionStateManager.access$000(ConnectionStateManager.java:43) at org.apache.curator.framework.state.ConnectionStateManager$1.call(ConnectionStateManager.java:111) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 26) Thread[id=19016, name=qtp344752888-19016, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 27) Thread[id=19058, name=Curator-Framework-0, state=WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.DelayQueue.take(DelayQueue.java:211) at java.util.concurrent.DelayQueue.take(DelayQueue.java:70) at org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:780) at org.apache.curator.framework.imps.CuratorFrameworkImpl.access$300(CuratorFrameworkImpl.java:62) at org.apache.curator.framework.imps.CuratorFrameworkImpl$4.call(CuratorFrameworkImpl.java:257) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 28) Thread[id=19009, name=qtp344752888-19009, state=RUNNABLE, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:745) 29) Thread[id=19033, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:745) 30) Thread[id=19034, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:745) 31) Thread[id=19045, name=jetty-launcher-3412-thread-1-SendThread(127.0.0.1:64286), state=RUNNABLE, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) 32) Thread[id=19007, name=jetty-launcher-3412-thread-1, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 33) Thread[id=19017, name=org.eclipse.jetty.server.session.HashSessionManager@7f65025aTimer, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 34) Thread[id=19010, name=qtp344752888-19010-acceptor-0@2460497b-ServerConnector@52e50448{SSL,[ssl, http/1.1]}{127.0.0.1:64288}, state=RUNNABLE, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:373) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:745) 35) Thread[id=19064, name=Curator-PathChildrenCache-2, state=WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 36) Thread[id=19021, name=qtp2110168235-19021, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 37) Thread[id=19044, name=jetty-launcher-3412-thread-2-EventThread, state=WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494) 38) Thread[id=19024, name=qtp2110168235-19024, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 39) Thread[id=18999, name=Thread-4385, state=WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at java.lang.Object.wait(Native Method) at java.lang.Thread.join(Thread.java:1249) at java.lang.Thread.join(Thread.java:1323) at org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:302) at org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:309) at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:490) 40) Thread[id=19008, name=jetty-launcher-3412-thread-2, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 41) Thread[id=19065, name=Curator-PathChildrenCache-3, state=WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 42) Thread[id=19055, name=jetty-launcher-3412-thread-1-SendThread(127.0.0.1:64286), state=RUNNABLE, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) 43) Thread[id=19014, name=qtp344752888-19014, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 44) Thread[id=19048, name=zkCallback-3423-thread-1, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 45) Thread[id=19046, name=jetty-launcher-3412-thread-1-EventThread, state=WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494) 46) Thread[id=19049, name=OverseerStateUpdate-97421845135032325-127.0.0.1:64288_solr-n_0000000000, state=TIMED_WAITING, group=Overseer state updater.] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:164) at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:138) at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:179) at java.lang.Thread.run(Thread.java:745) 47) Thread[id=19063, name=Curator-PathChildrenCache-1, state=WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 48) Thread[id=19053, name=Curator-ConnectionStateManager-0, state=WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:403) at org.apache.curator.framework.state.ConnectionStateManager.processEvents(ConnectionStateManager.java:245) at org.apache.curator.framework.state.ConnectionStateManager.access$000(ConnectionStateManager.java:43) at org.apache.curator.framework.state.ConnectionStateManager$1.call(ConnectionStateManager.java:111) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 49) Thread[id=19011, name=qtp344752888-19011, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 50) Thread[id=19070, name=pool-10-thread-1, state=WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 51) Thread[id=19003, name=ProcessThread(sid:0 cport:-1):, state=WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:120) 52) Thread[id=19012, name=qtp344752888-19012, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 53) Thread[id=19056, name=jetty-launcher-3412-thread-2-EventThread, state=WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494) 54) Thread[id=19067, name=Thread[Thread-4402,5,TGRP-TestSolrCloudWithDelegationTokens], state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at java.lang.Thread.sleep(Native Method) at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager$ExpiredTokenRemover.run(AbstractDelegationTokenSecretManager.java:657) at java.lang.Thread.run(Thread.java:745) 55) Thread[id=19022, name=qtp2110168235-19022, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 55 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithDelegationTokens:
1) Thread[id=19013, name=qtp344752888-19013, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
2) Thread[id=19025, name=qtp2110168235-19025, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
3) Thread[id=19018, name=qtp2110168235-19018, state=RUNNABLE, group=TGRP-TestSolrCloudWithDelegationTokens]
at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:745)
4) Thread[id=19043, name=jetty-launcher-3412-thread-2-SendThread(127.0.0.1:64286), state=RUNNABLE, group=TGRP-TestSolrCloudWithDelegationTokens]
at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
5) Thread[id=19054, name=jetty-launcher-3412-thread-2-SendThread(127.0.0.1:64286), state=RUNNABLE, group=TGRP-TestSolrCloudWithDelegationTokens]
at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
6) Thread[id=19051, name=OverseerHdfsCoreFailoverThread-97421845135032325-127.0.0.1:64288_solr-n_0000000000, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:139)
at java.lang.Thread.run(Thread.java:745)
7) Thread[id=19019, name=qtp2110168235-19019-acceptor-0@2e5b7fe0-ServerConnector@4b3e80d0{SSL,[ssl, http/1.1]}{127.0.0.1:64289}, state=RUNNABLE, group=TGRP-TestSolrCloudWithDelegationTokens]
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:373)
at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:745)
8) Thread[id=19035, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:745)
9) Thread[id=19000, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-TestSolrCloudWithDelegationTokens]
at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:178)
at java.lang.Thread.run(Thread.java:745)
10) Thread[id=19071, name=pool-11-thread-1, state=WAITING, group=TGRP-TestSolrCloudWithDelegationTokens]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(
[...truncated too long message...]
olExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 16) Thread[id=19055, name=jetty-launcher-3412-thread-1-SendThread(127.0.0.1:64286), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
[junit4] > 17) Thread[id=19059, name=Curator-Framework-0, state=WAITING, group=TGRP-TestSolrCloudWithDelegationTokens]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 18) Thread[id=19048, name=zkCallback-3423-thread-1, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 19) Thread[id=19052, name=Curator-ConnectionStateManager-0, state=WAITING, group=TGRP-TestSolrCloudWithDelegationTokens]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 20) Thread[id=19058, name=Curator-Framework-0, state=WAITING, group=TGRP-TestSolrCloudWithDelegationTokens]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 21) Thread[id=19063, name=Curator-PathChildrenCache-1, state=WAITING, group=TGRP-TestSolrCloudWithDelegationTokens]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 22) Thread[id=19053, name=Curator-ConnectionStateManager-0, state=WAITING, group=TGRP-TestSolrCloudWithDelegationTokens]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 23) Thread[id=19009, name=qtp344752888-19009, state=RUNNABLE, group=TGRP-TestSolrCloudWithDelegationTokens]
[junit4] > at sun.nio.ch.IOUtil.drain(Native Method)
[junit4] > at sun.nio.ch.KQueueSelectorImpl.updateSelectedKeys(KQueueSelectorImpl.java:181)
[junit4] > at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:122)
[junit4] > at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
[junit4] > at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
[junit4] > at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
[junit4] > at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 24) Thread[id=19070, name=pool-10-thread-1, state=WAITING, group=TGRP-TestSolrCloudWithDelegationTokens]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 25) Thread[id=19045, name=jetty-launcher-3412-thread-1-SendThread(127.0.0.1:64286), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] > 26) Thread[id=19067, name=Thread[Thread-4402,5,TGRP-TestSolrCloudWithDelegationTokens], state=TIMED_WAITING, group=TGRP-TestSolrCloudWithDelegationTokens]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager$ExpiredTokenRemover.run(AbstractDelegationTokenSecretManager.java:657)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > at __randomizedtesting.SeedInfo.seed([357F96641734EF54]:0)
[junit4] Completed [529/693 (1!)] on J0 in 56.76s, 0 tests, 1 failure, 5 errors <<< FAILURES!
[...truncated 62952 lines...]
[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3818
- Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3818/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC
9 tests failed.
FAILED: org.apache.solr.cloud.CustomCollectionTest.testCustomCollectionsAPI
Error Message:
Could not find collection : implicitcoll
Stack Trace:
org.apache.solr.common.SolrException: Could not find collection : implicitcoll
at __randomizedtesting.SeedInfo.seed([601FA41D176630EA:AFE2A762AFC8692]:0)
at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:194)
at org.apache.solr.cloud.SolrCloudTestCase.getCollectionState(SolrCloudTestCase.java:245)
at org.apache.solr.cloud.CustomCollectionTest.testCustomCollectionsAPI(CustomCollectionTest.java:68)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.CustomCollectionTest.testRouteFieldForHashRouter
Error Message:
Collection not found: routeFieldColl
Stack Trace:
org.apache.solr.common.SolrException: Collection not found: routeFieldColl
at __randomizedtesting.SeedInfo.seed([601FA41D176630EA:C8293AC08807DBB0]:0)
at org.apache.solr.client.solrj.impl.CloudSolrClient.getCollectionNames(CloudSolrClient.java:1376)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1072)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1042)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:232)
at org.apache.solr.cloud.CustomCollectionTest.testRouteFieldForHashRouter(CustomCollectionTest.java:166)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test
Error Message:
Could not find collection:collection2
Stack Trace:
java.lang.AssertionError: Could not find collection:collection2
at __randomizedtesting.SeedInfo.seed([601FA41D176630EA:E84B9BC7B99A5D12]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:159)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:144)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:139)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:856)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexingBatchPerRequestWithHttpSolrClient(FullSolrCloudDistribCmdsTest.java:620)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test(FullSolrCloudDistribCmdsTest.java:152)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.MultiThreadedOCPTest.test
Error Message:
Task 3001 did not complete, final state: FAILED expected same:<COMPLETED> was not:<FAILED>
Stack Trace:
java.lang.AssertionError: Task 3001 did not complete, final state: FAILED expected same:<COMPLETED> was not:<FAILED>
at __randomizedtesting.SeedInfo.seed([601FA41D176630EA:E84B9BC7B99A5D12]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotSame(Assert.java:641)
at org.junit.Assert.assertSame(Assert.java:580)
at org.apache.solr.cloud.MultiThreadedOCPTest.testDeduplicationOfSubmittedTasks(MultiThreadedOCPTest.java:250)
at org.apache.solr.cloud.MultiThreadedOCPTest.test(MultiThreadedOCPTest.java:64)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.OverseerRolesTest
Error Message:
java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:62163 within 30000 ms
Stack Trace:
org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:62163 within 30000 ms
at __randomizedtesting.SeedInfo.seed([601FA41D176630EA]:0)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:182)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:116)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:111)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:98)
at org.apache.solr.cloud.MiniSolrCloudCluster.waitForAllNodes(MiniSolrCloudCluster.java:260)
at org.apache.solr.cloud.MiniSolrCloudCluster.<init>(MiniSolrCloudCluster.java:254)
at org.apache.solr.cloud.SolrCloudTestCase$Builder.configure(SolrCloudTestCase.java:188)
at org.apache.solr.cloud.OverseerRolesTest.setupCluster(OverseerRolesTest.java:50)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:847)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:62163 within 30000 ms
at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:233)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:174)
... 31 more
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.OverseerRolesTest
Error Message:
ObjectTracker found 7 object(s) that were not released!!! [SolrZkClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:184) at org.apache.solr.cloud.ZkController.<init>(ZkController.java:296) at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:112) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:499) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:268) at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:195) at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:47) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:484) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:268) at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:195) at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:47) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:484) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:268) at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:90) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:490) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:268) at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:195) at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:47) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:484) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:268) at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:90) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:490) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:268) at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:90) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:490) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
java.lang.AssertionError: ObjectTracker found 7 object(s) that were not released!!! [SolrZkClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:184)
at org.apache.solr.cloud.ZkController.<init>(ZkController.java:296)
at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:112)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:499)
at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169)
at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376)
at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:268)
at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:195)
at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:47)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:484)
at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169)
at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376)
at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:268)
at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:195)
at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:47)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:484)
at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169)
at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376)
at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:268)
at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:90)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:490)
at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169)
at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376)
at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:268)
at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:195)
at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:47)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:484)
at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169)
at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376)
at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:268)
at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:90)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:490)
at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169)
at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376)
at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:268)
at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:90)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:490)
at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169)
at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376)
at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([601FA41D176630EA]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:289)
at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.OverseerRolesTest
Error Message:
51 threads leaked from SUITE scope at org.apache.solr.cloud.OverseerRolesTest: 1) Thread[id=18775, name=qtp740029689-18775, state=RUNNABLE, group=TGRP-OverseerRolesTest] at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:745) 2) Thread[id=18824, name=Connection evictor, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:745) 3) Thread[id=18802, name=qtp1251852185-18802, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 4) Thread[id=18764, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-OverseerRolesTest] at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:178) at java.lang.Thread.run(Thread.java:745) 5) Thread[id=18823, name=Connection evictor, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:745) 6) Thread[id=18836, name=OverseerHdfsCoreFailoverThread-97419033214976005-127.0.0.1:62165_solr-n_0000000000, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:139) at java.lang.Thread.run(Thread.java:745) 7) Thread[id=18791, name=qtp616498750-18791, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 8) Thread[id=18786, name=qtp740029689-18786, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 9) Thread[id=18825, name=Connection evictor, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:745) 10) Thread[id=18806, name=qtp1251852185-18806, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 11) Thread[id=18784, name=qtp1251852185-18784, state=RUNNABLE, group=TGRP-OverseerRolesTest] at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:745) 12) Thread[id=18808, name=qtp1251852185-18808, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 13) Thread[id=18778, name=qtp740029689-18778, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 14) Thread[id=18782, name=qtp616498750-18782-acceptor-0@ff80304-ServerConnector@3d0bd2fe{HTTP/1.1,[http/1.1]}{127.0.0.1:62165}, state=RUNNABLE, group=TGRP-OverseerRolesTest] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:373) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:745) 15) Thread[id=18804, name=qtp1251852185-18804, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 16) Thread[id=18787, name=qtp616498750-18787, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 17) Thread[id=18803, name=qtp1251852185-18803, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 18) Thread[id=18776, name=qtp740029689-18776-acceptor-0@303440f7-ServerConnector@45ae6903{HTTP/1.1,[http/1.1]}{127.0.0.1:62167}, state=RUNNABLE, group=TGRP-OverseerRolesTest] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:373) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:745) 19) Thread[id=18781, name=qtp616498750-18781, state=RUNNABLE, group=TGRP-OverseerRolesTest] at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:745) 20) Thread[id=18807, name=qtp1251852185-18807, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 21) Thread[id=18809, name=org.eclipse.jetty.server.session.HashSessionManager@39b16004Timer, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 22) Thread[id=18792, name=qtp740029689-18792, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 23) Thread[id=18793, name=qtp1153044004-18793-acceptor-0@27372cd1-ServerConnector@4698a7be{HTTP/1.1,[http/1.1]}{127.0.0.1:62166}, state=RUNNABLE, group=TGRP-OverseerRolesTest] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:373) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:745) 24) Thread[id=18777, name=qtp740029689-18777, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 25) Thread[id=18810, name=org.eclipse.jetty.server.session.HashSessionManager@6bc80fffTimer, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 26) Thread[id=18763, name=Thread-3886, state=WAITING, group=TGRP-OverseerRolesTest] at java.lang.Object.wait(Native Method) at java.lang.Thread.join(Thread.java:1249) at java.lang.Thread.join(Thread.java:1323) at org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:302) at org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:309) at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:490) 27) Thread[id=18796, name=qtp1153044004-18796, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 28) Thread[id=18797, name=qtp1153044004-18797, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 29) Thread[id=18834, name=OverseerStateUpdate-97419033214976005-127.0.0.1:62165_solr-n_0000000000, state=TIMED_WAITING, group=Overseer state updater.] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:164) at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:138) at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:179) at java.lang.Thread.run(Thread.java:745) 30) Thread[id=18819, name=Connection evictor, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:745) 31) Thread[id=18835, name=OverseerCollectionConfigSetProcessor-97419033214976005-127.0.0.1:62165_solr-n_0000000000, state=TIMED_WAITING, group=Overseer collection creation process.] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.apache.solr.cloud.DistributedQueue.peekElements(DistributedQueue.java:343) at org.apache.solr.cloud.OverseerTaskQueue.peekTopN(OverseerTaskQueue.java:235) at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:227) at java.lang.Thread.run(Thread.java:745) 32) Thread[id=18800, name=qtp1153044004-18800, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 33) Thread[id=18805, name=org.eclipse.jetty.server.session.HashSessionManager@7233a8f2Timer, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 34) Thread[id=18832, name=jetty-launcher-3604-thread-4-EventThread, state=WAITING, group=TGRP-OverseerRolesTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494) 35) Thread[id=18790, name=qtp616498750-18790, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 36) Thread[id=18783, name=qtp616498750-18783, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 37) Thread[id=18822, name=Connection evictor, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:745) 38) Thread[id=18765, name=SessionTracker, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at java.lang.Object.wait(Native Method) at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146) 39) Thread[id=18798, name=qtp1153044004-18798, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 40) Thread[id=18801, name=org.eclipse.jetty.server.session.HashSessionManager@4bc4165fTimer, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 41) Thread[id=18767, name=ProcessThread(sid:0 cport:-1):, state=WAITING, group=TGRP-OverseerRolesTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:120) 42) Thread[id=18794, name=qtp1251852185-18794-acceptor-0@6053f957-ServerConnector@126e2ccb{HTTP/1.1,[http/1.1]}{127.0.0.1:62168}, state=RUNNABLE, group=TGRP-OverseerRolesTest] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:373) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:745) 43) Thread[id=18779, name=qtp740029689-18779, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 44) Thread[id=18795, name=qtp1153044004-18795, state=TIMED_WAITING, group=TGRP-OverseerRolesTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
[...truncated too long message...]
lExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
[junit4] > at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 3) Thread[id=18801, name=org.eclipse.jetty.server.session.HashSessionManager@4bc4165fTimer, state=TIMED_WAITING, group=TGRP-OverseerRolesTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
[junit4] > at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
[junit4] > at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 4) Thread[id=18781, name=qtp616498750-18781, state=RUNNABLE, group=TGRP-OverseerRolesTest]
[junit4] > at sun.nio.ch.IOUtil.drain(Native Method)
[junit4] > at sun.nio.ch.KQueueSelectorImpl.updateSelectedKeys(KQueueSelectorImpl.java:181)
[junit4] > at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:122)
[junit4] > at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
[junit4] > at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
[junit4] > at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
[junit4] > at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 5) Thread[id=18809, name=org.eclipse.jetty.server.session.HashSessionManager@39b16004Timer, state=TIMED_WAITING, group=TGRP-OverseerRolesTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
[junit4] > at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
[junit4] > at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 6) Thread[id=18789, name=qtp1153044004-18789, state=RUNNABLE, group=TGRP-OverseerRolesTest]
[junit4] > at sun.nio.ch.KQueueArrayWrapper.interrupt(Native Method)
[junit4] > at sun.nio.ch.KQueueArrayWrapper.interrupt(KQueueArrayWrapper.java:203)
[junit4] > at sun.nio.ch.KQueueSelectorImpl.wakeup(KQueueSelectorImpl.java:257)
[junit4] > at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:213)
[junit4] > at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:219)
[junit4] > at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:116)
[junit4] > at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
[junit4] > at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
[junit4] > at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
[junit4] > at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 7) Thread[id=18805, name=org.eclipse.jetty.server.session.HashSessionManager@7233a8f2Timer, state=TIMED_WAITING, group=TGRP-OverseerRolesTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
[junit4] > at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
[junit4] > at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 8) Thread[id=18784, name=qtp1251852185-18784, state=RUNNABLE, group=TGRP-OverseerRolesTest]
[junit4] > at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
[junit4] > at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
[junit4] > at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117)
[junit4] > at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
[junit4] > at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
[junit4] > at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
[junit4] > at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 9) Thread[id=18831, name=jetty-launcher-3604-thread-4-SendThread(127.0.0.1:62163), state=TIMED_WAITING, group=TGRP-OverseerRolesTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] > at __randomizedtesting.SeedInfo.seed([601FA41D176630EA]:0)
[junit4] Completed [504/693 (5!)] on J1 in 118.10s, 0 tests, 1 failure, 3 errors <<< FAILURES!
[...truncated 63087 lines...]
[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3817
- Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3817/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
1 tests failed.
FAILED: org.apache.solr.cloud.PeerSyncReplicationTest.test
Error Message:
timeout waiting to see all nodes active
Stack Trace:
java.lang.AssertionError: timeout waiting to see all nodes active
at __randomizedtesting.SeedInfo.seed([BB43F3AEC31A5791:3317CC746DE63A69]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.PeerSyncReplicationTest.waitTillNodesActive(PeerSyncReplicationTest.java:326)
at org.apache.solr.cloud.PeerSyncReplicationTest.bringUpDeadNodeAndEnsureNoReplication(PeerSyncReplicationTest.java:277)
at org.apache.solr.cloud.PeerSyncReplicationTest.forceNodeFailureAndDoPeerSync(PeerSyncReplicationTest.java:259)
at org.apache.solr.cloud.PeerSyncReplicationTest.test(PeerSyncReplicationTest.java:138)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11557 lines...]
[junit4] Suite: org.apache.solr.cloud.PeerSyncReplicationTest
[junit4] 2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/init-core-data-001
[junit4] 2> 1418260 INFO (SUITE-PeerSyncReplicationTest-seed#[BB43F3AEC31A5791]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 1418264 INFO (SUITE-PeerSyncReplicationTest-seed#[BB43F3AEC31A5791]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN) w/ MAC_OS_X supressed clientAuth
[junit4] 2> 1418264 INFO (SUITE-PeerSyncReplicationTest-seed#[BB43F3AEC31A5791]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 1418266 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1418267 INFO (Thread-2038) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1418267 INFO (Thread-2038) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1418373 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.ZkTestServer start zk server on port:55545
[junit4] 2> 1418403 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1418410 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 1418413 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1418416 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1418419 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1418422 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1418425 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1418428 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1418430 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1418433 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1418436 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1419897 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/control-001/cores/collection1
[junit4] 2> 1419900 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1419902 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6b30fedb{/,null,AVAILABLE}
[junit4] 2> 1419902 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@14f47d4e{HTTP/1.1,[http/1.1]}{127.0.0.1:55548}
[junit4] 2> 1419902 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.e.j.s.Server Started @1425230ms
[junit4] 2> 1419903 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/tempDir-001/control/data, hostContext=/, hostPort=55548, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/control-001/cores}
[junit4] 2> 1419903 ERROR (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1419903 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 1419903 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1419903 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1419903 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-07T00:05:02.835Z
[junit4] 2> 1419908 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1419908 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/control-001/solr.xml
[junit4] 2> 1419925 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55545/solr
[junit4] 2> 1419971 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [n:127.0.0.1:55548_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:55548_
[junit4] 2> 1419974 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [n:127.0.0.1:55548_ ] o.a.s.c.Overseer Overseer (id=97414407861108740-127.0.0.1:55548_-n_0000000000) starting
[junit4] 2> 1420001 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [n:127.0.0.1:55548_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55548_
[junit4] 2> 1420004 INFO (zkCallback-1524-thread-1-processing-n:127.0.0.1:55548_) [n:127.0.0.1:55548_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1420551 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [n:127.0.0.1:55548_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/control-001/cores
[junit4] 2> 1420551 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [n:127.0.0.1:55548_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1420557 INFO (OverseerStateUpdate-97414407861108740-127.0.0.1:55548_-n_0000000000) [n:127.0.0.1:55548_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 1421599 INFO (coreLoadExecutor-4505-thread-1-processing-n:127.0.0.1:55548_) [n:127.0.0.1:55548_ c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1421622 INFO (coreLoadExecutor-4505-thread-1-processing-n:127.0.0.1:55548_) [n:127.0.0.1:55548_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1421787 WARN (coreLoadExecutor-4505-thread-1-processing-n:127.0.0.1:55548_) [n:127.0.0.1:55548_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1421790 INFO (coreLoadExecutor-4505-thread-1-processing-n:127.0.0.1:55548_) [n:127.0.0.1:55548_ c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 1421815 INFO (coreLoadExecutor-4505-thread-1-processing-n:127.0.0.1:55548_) [n:127.0.0.1:55548_ c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 1421816 INFO (coreLoadExecutor-4505-thread-1-processing-n:127.0.0.1:55548_) [n:127.0.0.1:55548_ c:control_collection x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/control-001/cores/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/control-001/cores/collection1/data/]
[junit4] 2> 1421816 INFO (coreLoadExecutor-4505-thread-1-processing-n:127.0.0.1:55548_) [n:127.0.0.1:55548_ c:control_collection x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e681b59
[junit4] 2> 1421819 INFO (coreLoadExecutor-4505-thread-1-processing-n:127.0.0.1:55548_) [n:127.0.0.1:55548_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=26, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.36771831036232683]
[junit4] 2> 1421830 WARN (coreLoadExecutor-4505-thread-1-processing-n:127.0.0.1:55548_) [n:127.0.0.1:55548_ c:control_collection x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1421900 INFO (coreLoadExecutor-4505-thread-1-processing-n:127.0.0.1:55548_) [n:127.0.0.1:55548_ c:control_collection x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1421900 INFO (coreLoadExecutor-4505-thread-1-processing-n:127.0.0.1:55548_) [n:127.0.0.1:55548_ c:control_collection x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1421901 INFO (coreLoadExecutor-4505-thread-1-processing-n:127.0.0.1:55548_) [n:127.0.0.1:55548_ c:control_collection x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1421901 INFO (coreLoadExecutor-4505-thread-1-processing-n:127.0.0.1:55548_) [n:127.0.0.1:55548_ c:control_collection x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1421901 INFO (coreLoadExecutor-4505-thread-1-processing-n:127.0.0.1:55548_) [n:127.0.0.1:55548_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=13, maxMergeAtOnceExplicit=33, maxMergedSegmentMB=78.880859375, floorSegmentMB=0.328125, forceMergeDeletesPctAllowed=5.292196089091311, segmentsPerTier=11.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.258532851684375
[junit4] 2> 1421902 INFO (coreLoadExecutor-4505-thread-1-processing-n:127.0.0.1:55548_) [n:127.0.0.1:55548_ c:control_collection x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7c246bb7[collection1] main]
[junit4] 2> 1421904 INFO (coreLoadExecutor-4505-thread-1-processing-n:127.0.0.1:55548_) [n:127.0.0.1:55548_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1421905 INFO (coreLoadExecutor-4505-thread-1-processing-n:127.0.0.1:55548_) [n:127.0.0.1:55548_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1421905 INFO (coreLoadExecutor-4505-thread-1-processing-n:127.0.0.1:55548_) [n:127.0.0.1:55548_ c:control_collection x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1421906 INFO (coreLoadExecutor-4505-thread-1-processing-n:127.0.0.1:55548_) [n:127.0.0.1:55548_ c:control_collection x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558630529591410688
[junit4] 2> 1421909 INFO (searcherExecutor-4506-thread-1-processing-n:127.0.0.1:55548_ x:collection1 c:control_collection) [n:127.0.0.1:55548_ c:control_collection x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7c246bb7[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1421919 INFO (coreZkRegister-4498-thread-1-processing-n:127.0.0.1:55548_ x:collection1 c:control_collection) [n:127.0.0.1:55548_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1421919 INFO (coreZkRegister-4498-thread-1-processing-n:127.0.0.1:55548_ x:collection1 c:control_collection) [n:127.0.0.1:55548_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1421919 INFO (coreZkRegister-4498-thread-1-processing-n:127.0.0.1:55548_ x:collection1 c:control_collection) [n:127.0.0.1:55548_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:55548/collection1/
[junit4] 2> 1421919 INFO (coreZkRegister-4498-thread-1-processing-n:127.0.0.1:55548_ x:collection1 c:control_collection) [n:127.0.0.1:55548_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1421919 INFO (coreZkRegister-4498-thread-1-processing-n:127.0.0.1:55548_ x:collection1 c:control_collection) [n:127.0.0.1:55548_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:55548/collection1/ has no replicas
[junit4] 2> 1421929 INFO (coreZkRegister-4498-thread-1-processing-n:127.0.0.1:55548_ x:collection1 c:control_collection) [n:127.0.0.1:55548_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:55548/collection1/ shard1
[junit4] 2> 1422048 INFO (coreZkRegister-4498-thread-1-processing-n:127.0.0.1:55548_ x:collection1 c:control_collection) [n:127.0.0.1:55548_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1422067 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1422069 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:55545/solr ready
[junit4] 2> 1422069 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 1422388 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/shard-1-001/cores/collection1
[junit4] 2> 1422390 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/shard-1-001
[junit4] 2> 1422390 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1422392 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7e5dfc53{/,null,AVAILABLE}
[junit4] 2> 1422392 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@54d057b5{HTTP/1.1,[http/1.1]}{127.0.0.1:55553}
[junit4] 2> 1422392 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.e.j.s.Server Started @1427720ms
[junit4] 2> 1422393 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=55553, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/shard-1-001/cores}
[junit4] 2> 1422393 ERROR (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1422394 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 1422394 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1422394 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1422394 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-07T00:05:05.326Z
[junit4] 2> 1422421 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1422421 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/shard-1-001/solr.xml
[junit4] 2> 1422441 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55545/solr
[junit4] 2> 1422460 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [n:127.0.0.1:55553_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1422470 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [n:127.0.0.1:55553_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55553_
[junit4] 2> 1422474 INFO (zkCallback-1524-thread-2-processing-n:127.0.0.1:55548_) [n:127.0.0.1:55548_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1422479 INFO (zkCallback-1533-thread-1-processing-n:127.0.0.1:55553_) [n:127.0.0.1:55553_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1422479 INFO (zkCallback-1528-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1422574 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [n:127.0.0.1:55553_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/shard-1-001/cores
[junit4] 2> 1422574 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [n:127.0.0.1:55553_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1422579 INFO (OverseerStateUpdate-97414407861108740-127.0.0.1:55548_-n_0000000000) [n:127.0.0.1:55548_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 1423615 INFO (coreLoadExecutor-4516-thread-1-processing-n:127.0.0.1:55553_) [n:127.0.0.1:55553_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1423637 INFO (coreLoadExecutor-4516-thread-1-processing-n:127.0.0.1:55553_) [n:127.0.0.1:55553_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1423801 WARN (coreLoadExecutor-4516-thread-1-processing-n:127.0.0.1:55553_) [n:127.0.0.1:55553_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1423804 INFO (coreLoadExecutor-4516-thread-1-processing-n:127.0.0.1:55553_) [n:127.0.0.1:55553_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 1423830 INFO (coreLoadExecutor-4516-thread-1-processing-n:127.0.0.1:55553_) [n:127.0.0.1:55553_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1423831 INFO (coreLoadExecutor-4516-thread-1-processing-n:127.0.0.1:55553_) [n:127.0.0.1:55553_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/shard-1-001/cores/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/shard-1-001/cores/collection1/data/]
[junit4] 2> 1423831 INFO (coreLoadExecutor-4516-thread-1-processing-n:127.0.0.1:55553_) [n:127.0.0.1:55553_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e681b59
[junit4] 2> 1423834 INFO (coreLoadExecutor-4516-thread-1-processing-n:127.0.0.1:55553_) [n:127.0.0.1:55553_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=26, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.36771831036232683]
[junit4] 2> 1423850 WARN (coreLoadExecutor-4516-thread-1-processing-n:127.0.0.1:55553_) [n:127.0.0.1:55553_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1423925 INFO (coreLoadExecutor-4516-thread-1-processing-n:127.0.0.1:55553_) [n:127.0.0.1:55553_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1423925 INFO (coreLoadExecutor-4516-thread-1-processing-n:127.0.0.1:55553_) [n:127.0.0.1:55553_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1423927 INFO (coreLoadExecutor-4516-thread-1-processing-n:127.0.0.1:55553_) [n:127.0.0.1:55553_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1423927 INFO (coreLoadExecutor-4516-thread-1-processing-n:127.0.0.1:55553_) [n:127.0.0.1:55553_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1423928 INFO (coreLoadExecutor-4516-thread-1-processing-n:127.0.0.1:55553_) [n:127.0.0.1:55553_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=13, maxMergeAtOnceExplicit=33, maxMergedSegmentMB=78.880859375, floorSegmentMB=0.328125, forceMergeDeletesPctAllowed=5.292196089091311, segmentsPerTier=11.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.258532851684375
[junit4] 2> 1423930 INFO (coreLoadExecutor-4516-thread-1-processing-n:127.0.0.1:55553_) [n:127.0.0.1:55553_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@9c6f1ac[collection1] main]
[junit4] 2> 1423931 INFO (coreLoadExecutor-4516-thread-1-processing-n:127.0.0.1:55553_) [n:127.0.0.1:55553_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1423932 INFO (coreLoadExecutor-4516-thread-1-processing-n:127.0.0.1:55553_) [n:127.0.0.1:55553_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1423932 INFO (coreLoadExecutor-4516-thread-1-processing-n:127.0.0.1:55553_) [n:127.0.0.1:55553_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1423933 INFO (searcherExecutor-4517-thread-1-processing-n:127.0.0.1:55553_ x:collection1 c:collection1) [n:127.0.0.1:55553_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@9c6f1ac[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1423934 INFO (coreLoadExecutor-4516-thread-1-processing-n:127.0.0.1:55553_) [n:127.0.0.1:55553_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558630531717922816
[junit4] 2> 1423961 INFO (coreZkRegister-4511-thread-1-processing-n:127.0.0.1:55553_ x:collection1 c:collection1) [n:127.0.0.1:55553_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1423961 INFO (coreZkRegister-4511-thread-1-processing-n:127.0.0.1:55553_ x:collection1 c:collection1) [n:127.0.0.1:55553_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1423962 INFO (coreZkRegister-4511-thread-1-processing-n:127.0.0.1:55553_ x:collection1 c:collection1) [n:127.0.0.1:55553_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:55553/collection1/
[junit4] 2> 1423962 INFO (coreZkRegister-4511-thread-1-processing-n:127.0.0.1:55553_ x:collection1 c:collection1) [n:127.0.0.1:55553_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1423962 INFO (coreZkRegister-4511-thread-1-processing-n:127.0.0.1:55553_ x:collection1 c:collection1) [n:127.0.0.1:55553_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:55553/collection1/ has no replicas
[junit4] 2> 1423970 INFO (coreZkRegister-4511-thread-1-processing-n:127.0.0.1:55553_ x:collection1 c:collection1) [n:127.0.0.1:55553_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:55553/collection1/ shard1
[junit4] 2> 1424130 INFO (coreZkRegister-4511-thread-1-processing-n:127.0.0.1:55553_ x:collection1 c:collection1) [n:127.0.0.1:55553_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1424374 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/shard-2-001/cores/collection1
[junit4] 2> 1424376 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/shard-2-001
[junit4] 2> 1424376 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1424378 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@572034b7{/,null,AVAILABLE}
[junit4] 2> 1424378 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@500fc684{HTTP/1.1,[http/1.1]}{127.0.0.1:55557}
[junit4] 2> 1424379 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.e.j.s.Server Started @1429706ms
[junit4] 2> 1424379 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/, hostPort=55557, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/shard-2-001/cores}
[junit4] 2> 1424380 ERROR (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1424380 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 1424380 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1424380 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1424381 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-07T00:05:07.313Z
[junit4] 2> 1424385 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1424385 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/shard-2-001/solr.xml
[junit4] 2> 1424413 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55545/solr
[junit4] 2> 1424432 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [n:127.0.0.1:55557_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1424443 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [n:127.0.0.1:55557_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55557_
[junit4] 2> 1424447 INFO (zkCallback-1528-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1424448 INFO (zkCallback-1539-thread-1-processing-n:127.0.0.1:55557_) [n:127.0.0.1:55557_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1424447 INFO (zkCallback-1533-thread-1-processing-n:127.0.0.1:55553_) [n:127.0.0.1:55553_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1424447 INFO (zkCallback-1524-thread-3-processing-n:127.0.0.1:55548_) [n:127.0.0.1:55548_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1424562 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [n:127.0.0.1:55557_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/shard-2-001/cores
[junit4] 2> 1424562 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [n:127.0.0.1:55557_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1424566 INFO (OverseerStateUpdate-97414407861108740-127.0.0.1:55548_-n_0000000000) [n:127.0.0.1:55548_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 1425597 INFO (coreLoadExecutor-4527-thread-1-processing-n:127.0.0.1:55557_) [n:127.0.0.1:55557_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1425619 INFO (coreLoadExecutor-4527-thread-1-processing-n:127.0.0.1:55557_) [n:127.0.0.1:55557_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1425783 WARN (coreLoadExecutor-4527-thread-1-processing-n:127.0.0.1:55557_) [n:127.0.0.1:55557_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1425785 INFO (coreLoadExecutor-4527-thread-1-processing-n:127.0.0.1:55557_) [n:127.0.0.1:55557_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 1425810 INFO (coreLoadExecutor-4527-thread-1-processing-n:127.0.0.1:55557_) [n:127.0.0.1:55557_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1425810 INFO (coreLoadExecutor-4527-thread-1-processing-n:127.0.0.1:55557_) [n:127.0.0.1:55557_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/shard-2-001/cores/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/shard-2-001/cores/collection1/data/]
[junit4] 2> 1425811 INFO (coreLoadExecutor-4527-thread-1-processing-n:127.0.0.1:55557_) [n:127.0.0.1:55557_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e681b59
[junit4] 2> 1425817 INFO (coreLoadExecutor-4527-thread-1-processing-n:127.0.0.1:55557_) [n:127.0.0.1:55557_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=26, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.36771831036232683]
[junit4] 2> 1425828 WARN (coreLoadExecutor-4527-thread-1-processing-n:127.0.0.1:55557_) [n:127.0.0.1:55557_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1425954 INFO (coreLoadExecutor-4527-thread-1-processing-n:127.0.0.1:55557_) [n:127.0.0.1:55557_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1425954 INFO (coreLoadExecutor-4527-thread-1-processing-n:127.0.0.1:55557_) [n:127.0.0.1:55557_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1425955 INFO (coreLoadExecutor-4527-thread-1-processing-n:127.0.0.1:55557_) [n:127.0.0.1:55557_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1425955 INFO (coreLoadExecutor-4527-thread-1-processing-n:127.0.0.1:55557_) [n:127.0.0.1:55557_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1425956 INFO (coreLoadExecutor-4527-thread-1-processing-n:127.0.0.1:55557_) [n:127.0.0.1:55557_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=13, maxMergeAtOnceExplicit=33, maxMergedSegmentMB=78.880859375, floorSegmentMB=0.328125, forceMergeDeletesPctAllowed=5.292196089091311, segmentsPerTier=11.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.258532851684375
[junit4] 2> 1425964 INFO (coreLoadExecutor-4527-thread-1-processing-n:127.0.0.1:55557_) [n:127.0.0.1:55557_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@9b41bb7[collection1] main]
[junit4] 2> 1425966 INFO (coreLoadExecutor-4527-thread-1-processing-n:127.0.0.1:55557_) [n:127.0.0.1:55557_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1425966 INFO (coreLoadExecutor-4527-thread-1-processing-n:127.0.0.1:55557_) [n:127.0.0.1:55557_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1425967 INFO (coreLoadExecutor-4527-thread-1-processing-n:127.0.0.1:55557_) [n:127.0.0.1:55557_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1425968 INFO (coreLoadExecutor-4527-thread-1-processing-n:127.0.0.1:55557_) [n:127.0.0.1:55557_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558630533850726400
[junit4] 2> 1425971 INFO (searcherExecutor-4528-thread-1-processing-n:127.0.0.1:55557_ x:collection1 c:collection1) [n:127.0.0.1:55557_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@9b41bb7[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1425974 INFO (coreZkRegister-4522-thread-1-processing-n:127.0.0.1:55557_ x:collection1 c:collection1) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkController Core needs to recover:collection1
[junit4] 2> 1425975 INFO (updateExecutor-1536-thread-1-processing-n:127.0.0.1:55557_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 1425975 INFO (recoveryExecutor-1537-thread-1-processing-n:127.0.0.1:55557_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 1425975 INFO (recoveryExecutor-1537-thread-1-processing-n:127.0.0.1:55557_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 1425975 INFO (recoveryExecutor-1537-thread-1-processing-n:127.0.0.1:55557_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1]
[junit4] 2> 1425976 INFO (recoveryExecutor-1537-thread-1-processing-n:127.0.0.1:55557_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 1425976 INFO (recoveryExecutor-1537-thread-1-processing-n:127.0.0.1:55557_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core [collection1] as recovering, leader is [http://127.0.0.1:55553/collection1/] and I am [http://127.0.0.1:55557/collection1/]
[junit4] 2> 1425983 INFO (recoveryExecutor-1537-thread-1-processing-n:127.0.0.1:55557_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:55553]; [WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:55557_&coreNodeName=core_node2&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 1425986 INFO (qtp590866229-9095) [n:127.0.0.1:55553_ ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 1425987 INFO (qtp590866229-9095) [n:127.0.0.1:55553_ ] o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 (shard1 of collection1) have state: recovering
[junit4] 2> 1425987 INFO (qtp590866229-9095) [n:127.0.0.1:55553_ ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:55557_, coreNodeName=core_node2, onlyIfActiveCheckResult=false, nodeProps: core_node2:{"core":"collection1","base_url":"http://127.0.0.1:55557","node_name":"127.0.0.1:55557_","state":"down"}
[junit4] 2> 1426408 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/shard-3-001/cores/collection1
[junit4] 2> 1426410 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/shard-3-001
[junit4] 2> 1426413 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1426415 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6225f65d{/,null,AVAILABLE}
[junit4] 2> 1426416 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@25edd78{HTTP/1.1,[http/1.1]}{127.0.0.1:55562}
[junit4] 2> 1426416 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.e.j.s.Server Started @1431744ms
[junit4] 2> 1426416 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/tempDir-001/jetty3, solrconfig=solrconfig.xml, hostContext=/, hostPort=55562, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/shard-3-001/cores}
[junit4] 2> 1426416 ERROR (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1426417 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 1426417 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1426417 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1426417 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-07T00:05:09.349Z
[junit4] 2> 1426436 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1426436 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/shard-3-001/solr.xml
[junit4] 2> 1426454 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55545/solr
[junit4] 2> 1426473 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [n:127.0.0.1:55562_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1426485 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [n:127.0.0.1:55562_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55562_
[junit4] 2> 1426489 INFO (zkCallback-1528-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1426490 INFO (zkCallback-1546-thread-1-processing-n:127.0.0.1:55562_) [n:127.0.0.1:55562_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1426489 INFO (zkCallback-1533-thread-1-processing-n:127.0.0.1:55553_) [n:127.0.0.1:55553_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1426489 INFO (zkCallback-1524-thread-2-processing-n:127.0.0.1:55548_) [n:127.0.0.1:55548_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1426490 INFO (zkCallback-1539-thread-1-processing-n:127.0.0.1:55557_) [n:127.0.0.1:55557_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1426645 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [n:127.0.0.1:55562_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/shard-3-001/cores
[junit4] 2> 1426646 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [n:127.0.0.1:55562_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1426648 INFO (OverseerStateUpdate-97414407861108740-127.0.0.1:55548_-n_0000000000) [n:127.0.0.1:55548_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 1426991 INFO (qtp590866229-9095) [n:127.0.0.1:55553_ ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:55557_, coreNodeName=core_node2, onlyIfActiveCheckResult=false, nodeProps: core_node2:{"core":"collection1","base_url":"http://127.0.0.1:55557","node_name":"127.0.0.1:55557_","state":"recovering"}
[junit4] 2> 1426991 INFO (qtp590866229-9095) [n:127.0.0.1:55553_ ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 1426991 INFO (qtp590866229-9095) [n:127.0.0.1:55553_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:55557_&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node2&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1005
[junit4] 2> 1427677 INFO (coreLoadExecutor-4538-thread-1-processing-n:127.0.0.1:55562_) [n:127.0.0.1:55562_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1427702 INFO (coreLoadExecutor-4538-thread-1-processing-n:127.0.0.1:55562_) [n:127.0.0.1:55562_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1427879 WARN (coreLoadExecutor-4538-thread-1-processing-n:127.0.0.1:55562_) [n:127.0.0.1:55562_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1427881 INFO (coreLoadExecutor-4538-thread-1-processing-n:127.0.0.1:55562_) [n:127.0.0.1:55562_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 1427909 INFO (coreLoadExecutor-4538-thread-1-processing-n:127.0.0.1:55562_) [n:127.0.0.1:55562_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1427909 INFO (coreLoadExecutor-4538-thread-1-processing-n:127.0.0.1:55562_) [n:127.0.0.1:55562_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/shard-3-001/cores/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001/shard-3-001/cores/collection1/data/]
[junit4] 2> 1427910 INFO (coreLoadExecutor-4538-thread-1-processing-n:127.0.0.1:55562_) [n:127.0.0.1:55562_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e681b59
[junit4] 2> 1427915 INFO (coreLoadExecutor-4538-thread-1-processing-n:127.0.0.1:55562_) [n:127.0.0.1:55562_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=26, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.36771831036232683]
[junit4] 2> 1427927 WARN (coreLoadExecutor-4538-thread-1-processing-n:127.0.0.1:55562_) [n:127.0.0.1:55562_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1428044 INFO (coreLoadExecutor-4538-thread-1-processing-n:127.0.0.1:55562_) [n:127.0.0.1:55562_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1428044 INFO (coreLoadExecutor-4538-thread-1-processing-n:127.0.0.1:55562_) [n:127.0.0.1:55562_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1428045 INFO (coreLoadExecutor-4538-thread-1-processing-n:127.0.0.1:55562_) [n:127.0.0.1:55562_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1428046 INFO (coreLoadExecutor-4538-thread-1-processing-n:127.0.0.1:55562_) [n:127.0.0.1:55562_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1428047 INFO (coreLoadExecutor-4538-thread-1-processing-n:127.0.0.1:55562_) [n:127.0.0.1:55562_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=13, maxMergeAtOnceExplicit=33, maxMergedSegmentMB=78.880859375, floorSegmentMB=0.328125, forceMergeDeletesPctAllowed=5.292196089091311, segmentsPerTier=11.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.258532851684375
[junit4] 2> 1428048 INFO (coreLoadExecutor-4538-thread-1-processing-n:127.0.0.1:55562_) [n:127.0.0.1:55562_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4409b887[collection1] main]
[junit4] 2> 1428050 INFO (coreLoadExecutor-4538-thread-1-processing-n:127.0.0.1:55562_) [n:127.0.0.1:55562_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1428051 INFO (coreLoadExecutor-4538-thread-1-processing-n:127.0.0.1:55562_) [n:127.0.0.1:55562_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1428051 INFO (coreLoadExecutor-4538-thread-1-processing-n:127.0.0.1:55562_) [n:127.0.0.1:55562_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1428053 INFO (searcherExecutor-4539-thread-1-processing-n:127.0.0.1:55562_ x:collection1 c:collection1) [n:127.0.0.1:55562_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4409b887[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1428053 INFO (coreLoadExecutor-4538-thread-1-processing-n:127.0.0.1:55562_) [n:127.0.0.1:55562_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558630536037007360
[junit4] 2> 1428060 INFO (coreZkRegister-4533-thread-1-processing-n:127.0.0.1:55562_ x:collection1 c:collection1) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.ZkController Core needs to recover:collection1
[junit4] 2> 1428060 INFO (updateExecutor-1543-thread-1-processing-n:127.0.0.1:55562_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 1428061 INFO (recoveryExecutor-1544-thread-1-processing-n:127.0.0.1:55562_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 1428061 INFO (recoveryExecutor-1544-thread-1-processing-n:127.0.0.1:55562_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 1428061 INFO (recoveryExecutor-1544-thread-1-processing-n:127.0.0.1:55562_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1]
[junit4] 2> 1428062 INFO (recoveryExecutor-1544-thread-1-processing-n:127.0.0.1:55562_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 1428062 INFO (recoveryExecutor-1544-thread-1-processing-n:127.0.0.1:55562_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core [collection1] as recovering, leader is [http://127.0.0.1:55553/collection1/] and I am [http://127.0.0.1:55562/collection1/]
[junit4] 2> 1428070 INFO (recoveryExecutor-1544-thread-1-processing-n:127.0.0.1:55562_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:55553]; [WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:55562_&coreNodeName=core_node3&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 1428071 INFO (qtp590866229-9099) [n:127.0.0.1:55553_ ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 1428072 INFO (qtp590866229-9099) [n:127.0.0.1:55553_ ] o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 (shard1 of collection1) have state: recovering
[junit4] 2> 1428072 INFO (qtp590866229-9099) [n:127.0.0.1:55553_ ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:55562_, coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: core_node3:{"core":"collection1","base_url":"http://127.0.0.1:55562","node_name":"127.0.0.1:55562_","state":"down"}
[junit4] 2> 1428128 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 1428129 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Wait for recoveries to finish - wait 30 for each attempt
[junit4] 2> 1428129 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):30
[junit4] 2> 1429075 INFO (qtp590866229-9099) [n:127.0.0.1:55553_ ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:55562_, coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: core_node3:{"core":"collection1","base_url":"http://127.0.0.1:55562","node_name":"127.0.0.1:55562_","state":"recovering"}
[junit4] 2> 1429075 INFO (qtp590866229-9099) [n:127.0.0.1:55553_ ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 1429075 INFO (qtp590866229-9099) [n:127.0.0.1:55553_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:55562_&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node3&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1003
[junit4] 2> 1433997 INFO (recoveryExecutor-1537-thread-1-processing-n:127.0.0.1:55557_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:55553/collection1/] - recoveringAfterStartup=[true]
[junit4] 2> 1433997 INFO (recoveryExecutor-1537-thread-1-processing-n:127.0.0.1:55557_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.PeerSync PeerSync: core=collection1 url=http://127.0.0.1:55557 START replicas=[http://127.0.0.1:55553/collection1/] nUpdates=1000
[junit4] 2> 1434000 INFO (qtp590866229-9096) [n:127.0.0.1:55553_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 1434000 INFO (qtp590866229-9096) [n:127.0.0.1:55553_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/get params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 1434001 INFO (recoveryExecutor-1537-thread-1-processing-n:127.0.0.1:55557_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 1434001 INFO (recoveryExecutor-1537-thread-1-processing-n:127.0.0.1:55557_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.PeerSync We are already in sync. No need to do a PeerSync
[junit4] 2> 1434001 INFO (recoveryExecutor-1537-thread-1-processing-n:127.0.0.1:55557_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1434001 INFO (recoveryExecutor-1537-thread-1-processing-n:127.0.0.1:55557_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 1434001 INFO (recoveryExecutor-1537-thread-1-processing-n:127.0.0.1:55557_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1434001 INFO (recoveryExecutor-1537-thread-1-processing-n:127.0.0.1:55557_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy PeerSync stage of recovery was successful.
[junit4] 2> 1434001 INFO (recoveryExecutor-1537-thread-1-processing-n:127.0.0.1:55557_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Replaying updates buffered during PeerSync.
[junit4] 2> 1434001 INFO (recoveryExecutor-1537-thread-1-processing-n:127.0.0.1:55557_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy No replay needed.
[junit4] 2> 1434001 INFO (recoveryExecutor-1537-thread-1-processing-n:127.0.0.1:55557_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Registering as Active after recovery.
[junit4] 2> 1436080 INFO (recoveryExecutor-1544-thread-1-processing-n:127.0.0.1:55562_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:55553/collection1/] - recoveringAfterStartup=[true]
[junit4] 2> 1436081 INFO (recoveryExecutor-1544-thread-1-processing-n:127.0.0.1:55562_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.PeerSync PeerSync: core=collection1 url=http://127.0.0.1:55562 START replicas=[http://127.0.0.1:55553/collection1/] nUpdates=1000
[junit4] 2> 1436083 INFO (qtp590866229-9097) [n:127.0.0.1:55553_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 1436084 INFO (qtp590866229-9097) [n:127.0.0.1:55553_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/get params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1436085 INFO (recoveryExecutor-1544-thread-1-processing-n:127.0.0.1:55562_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 1436085 INFO (recoveryExecutor-1544-thread-1-processing-n:127.0.0.1:55562_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.PeerSync We are already in sync. No need to do a PeerSync
[junit4] 2> 1436085 INFO (recoveryExecutor-1544-thread-1-processing-n:127.0.0.1:55562_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1436085 INFO (recoveryExecutor-1544-thread-1-processing-n:127.0.0.1:55562_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 1436086 INFO (recoveryExecutor-1544-thread-1-processing-n:127.0.0.1:55562_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1436086 INFO (recoveryExecutor-1544-thread-1-processing-n:127.0.0.1:55562_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy PeerSync stage of recovery was successful.
[junit4] 2> 1436086 INFO (recoveryExecutor-1544-thread-1-processing-n:127.0.0.1:55562_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Replaying updates buffered during PeerSync.
[junit4] 2> 1436086 INFO (recoveryExecutor-1544-thread-1-processing-n:127.0.0.1:55562_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy No replay needed.
[junit4] 2> 1436086 INFO (recoveryExecutor-1544-thread-1-processing-n:127.0.0.1:55562_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Registering as Active after recovery.
[junit4] 2> 1436186 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: collection1
[junit4] 2> 1436193 INFO (qtp1536541908-9058) [n:127.0.0.1:55548_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1436193 INFO (qtp1536541908-9058) [n:127.0.0.1:55548_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 1436194 INFO (qtp1536541908-9058) [n:127.0.0.1:55548_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1436194 INFO (qtp1536541908-9058) [n:127.0.0.1:55548_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 2
[junit4] 2> 1436198 INFO (qtp590866229-9095) [n:127.0.0.1:55553_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1436198 INFO (qtp590866229-9095) [n:127.0.0.1:55553_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 1436198 INFO (qtp590866229-9095) [n:127.0.0.1:55553_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1436199 INFO (qtp590866229-9095) [n:127.0.0.1:55553_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:55553/collection1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 0
[junit4] 2> 1436203 INFO (qtp2051013821-9123) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1436203 INFO (qtp2051013821-9123) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 1436203 INFO (qtp1738667756-9158) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1436203 INFO (qtp1738667756-9158) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 1436203 INFO (qtp2051013821-9123) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1436204 INFO (qtp1738667756-9158) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1436204 INFO (qtp2051013821-9123) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:55553/collection1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 1
[junit4] 2> 1436204 INFO (qtp1738667756-9158) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:55553/collection1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 0
[junit4] 2> 1436204 INFO (qtp590866229-9100) [n:127.0.0.1:55553_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 8
[junit4] 2> 1436207 INFO (qtp590866229-9098) [n:127.0.0.1:55553_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&distrib=false&tests=checkShardConsistency&rows=0&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1436209 INFO (qtp2051013821-9125) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&distrib=false&tests=checkShardConsistency&rows=0&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1436210 INFO (qtp1738667756-9160) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&distrib=false&tests=checkShardConsistency&rows=0&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1438216 INFO (qtp1536541908-9059) [n:127.0.0.1:55548_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{deleteByQuery=*:* (-1558630546690539520)} 0 3
[junit4] 2> 1438228 INFO (qtp1738667756-9161) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1558630546694733824&distrib.from=http://127.0.0.1:55553/collection1/&wt=javabin&version=2}{deleteByQuery=*:* (-1558630546694733824)} 0 6
[junit4] 2> 1438229 INFO (qtp2051013821-9126) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1558630546694733824&distrib.from=http://127.0.0.1:55553/collection1/&wt=javabin&version=2}{deleteByQuery=*:* (-1558630546694733824)} 0 7
[junit4] 2> 1438230 INFO (qtp590866229-9097) [n:127.0.0.1:55553_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{deleteByQuery=*:* (-1558630546694733824)} 0 12
[junit4] 2> 1438245 INFO (qtp2051013821-9127) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:55553/collection1/&wt=javabin&version=2}{add=[0 (1558630546716753920)]} 0 1
[junit4] 2> 1438245 INFO (qtp1738667756-9162) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:55553/collection1/&wt=javabin&version=2}{add=[0 (1558630546716753920)]} 0 1
[junit4] 2> 1438246 INFO (qtp590866229-9099) [n:127.0.0.1:55553_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[0 (1558630546716753920)]} 0 8
[junit4] 2> 1438249 INFO (qtp2051013821-9128) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:55553/collection1/&wt=javabin&version=2}{add=[1 (1558630546726191104)]} 0 0
[junit4] 2> 1438250 INFO (qtp1738667756-9163) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:55553/collection1/&wt=javabin&version=2}{add=[1 (1558630546726191104)]} 0 0
[junit4] 2> 1438251 INFO (qtp590866229-9093) [n:127.0.0.1:55553_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[1 (1558630546726191104)]} 0 3
[junit4] 2> 1438254 INFO (qtp2051013821-9128) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:55553/collection1/&wt=javabin&version=2}{add=[2 (1558630546732482560)]} 0 0
[junit4] 2> 1438256 INFO (qtp1738667756-9163) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:55553/collection1/&wt=javabin&version=2}{add=[2 (1558630546732482560)]} 0 0
[junit4] 2> 1438256 INFO (qtp590866229-9095) [n:127.0.0.1:55553_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[2 (1558630546732482560)]} 0 3
[junit4] 2> 1438259 INFO (qtp2051013821-9128) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:55553/collection1/&wt=javabin&version=2}{add=[3 (1558630546736676864)]} 0 0
[junit4] 2> 1438260 INFO (qtp1738667756-9163) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:55553/collection1/&wt=javabin&version=2}{add=[3 (1558630546736676864)]} 0 0
[junit4] 2> 1438260 INFO (qtp590866229-9100) [n:127.0.0.1:55553_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[3 (1558630546736676864)]} 0 3
[junit4] 2> 1438264 INFO (qtp2051013821-9128) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:55553/collection1/&wt=javabin&version=2}{add=[4 (1558630546741919744)]} 0 0
[junit4] 2> 1438265 INFO (qtp1738667756-9163) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:55553/collection1/&wt=javabin&version=2}{add=[4 (1558630546741919744)]} 0 0
[junit4] 2> 1438265 INFO (qtp590866229-9096) [n:127.0.0.1:55553_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[4 (1558630546741919744)]} 0 3
[junit4] 2> 1438271 INFO (qtp2051013821-9128) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:55553/collection1/&wt=javabin&version=2}{add=[5 (1558630546749259776)]} 0 0
[junit4] 2> 1438272 INFO (qtp1738667756-9163) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:55553/collection1/&wt=javabin&version=2}{add=[5 (1558630546749259776)]} 0 0
[junit4] 2> 1438273 INFO (qtp590866229-9098) [n:127.0.0.1:55553_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[5 (1558630546749259776)]} 0 4
[junit4] 2> 1438278 INFO (qtp2051013821-9128) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:55553/collection1/&wt=javabin&version=2}{add=[6 (1558630546755551232)]} 0 0
[junit4] 2> 1438279 INFO (qtp1738667756-9163) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:55553/collection1/&wt=javabin&version=2}{add=[6 (1558630546755551232)]} 0 0
[junit4] 2> 1438279 INFO (qtp590866229-9097) [n:127.0.0.1:55553_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[6 (1558630546755551232)]} 0 4
[junit4] 2> 1438283 INFO (qtp2051013821-9128) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:55553/collection1/&wt=javabin&version=2}{add=[7 (1558630546761842688)]} 0 0
[junit4] 2> 1438283 INFO (qtp1738667756-9162) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:55553/collection1/&wt=javabin&version=2}{add=[7 (1558630546761842688)]} 0 0
[junit4] 2> 1438284 INFO (qtp590866229-9099) [n:127.0.0.1:55553_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[7 (1558630546761842688)]} 0 3
[junit4] 2> 1438286 INFO (qtp2051013821-9128) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:55553/collection1/&wt=javabin&version=2}{add=[8 (1558630546766036992)]} 0 0
[junit4] 2> 1438286 INFO (qtp1738667756-9156) [n:127.0.0.1:55562_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:55553/collection1/&wt=javabin&version=2}{add=[8 (1558630546766036992)]} 0 0
[junit4] 2> 1438289 INFO (qtp590866229-9093) [n:127.0.0.1:55553_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collect
[...truncated too long message...]
5791]) [ ] o.a.s.c.Overseer Overseer (id=97414407861108753-127.0.0.1:55557_-n_0000000004) closing
[junit4] 2> 1629312 INFO (OverseerStateUpdate-97414407861108753-127.0.0.1:55557_-n_0000000004) [n:127.0.0.1:55557_ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:55557_
[junit4] 2> 1629312 INFO (zkCallback-1553-thread-2-processing-n:127.0.0.1:55557_) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@79d18ec9
[junit4] 2> 1629313 WARN (zkCallback-1553-thread-2-processing-n:127.0.0.1:55557_) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Stopping recovery for core=[collection1] coreNodeName=[core_node2]
[junit4] 2> 1629314 WARN (zkCallback-1553-thread-3-processing-n:127.0.0.1:55557_) [n:127.0.0.1:55557_ ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 1629377 INFO (zkCallback-1553-thread-2-processing-n:127.0.0.1:55557_) [n:127.0.0.1:55557_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection1
[junit4] 2> 1629378 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 1629379 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@7e3958cc{HTTP/1.1,[http/1.1]}{127.0.0.1:55557}
[junit4] 2> 1629380 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2096334a{/,null,UNAVAILABLE}
[junit4] 2> 1629381 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.ChaosMonkey monkey: stop shard! 55562
[junit4] 2> 1629383 INFO (TEST-PeerSyncReplicationTest.test-seed#[BB43F3AEC31A5791]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:55545 55545
[junit4] 2> 1629428 INFO (Thread-2038) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:55545 55545
[junit4] 2> 1631869 WARN (Thread-2038) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/aliases.json
[junit4] 2> 6 /solr/clusterprops.json
[junit4] 2> 5 /solr/security.json
[junit4] 2> 5 /solr/configs/conf1
[junit4] 2> 4 /solr/collections/collection1/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/clusterstate.json
[junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/97414407861108744-core_node1-n_0000000000
[junit4] 2> 2 /solr/overseer_elect/election/97414407861108744-127.0.0.1:55553_-n_0000000001
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 207 /solr/overseer/collection-queue-work
[junit4] 2> 43 /solr/overseer/queue
[junit4] 2> 6 /solr/collections
[junit4] 2> 6 /solr/overseer/queue-work
[junit4] 2> 5 /solr/live_nodes
[junit4] 2>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=PeerSyncReplicationTest -Dtests.method=test -Dtests.seed=BB43F3AEC31A5791 -Dtests.slow=true -Dtests.locale=zh-SG -Dtests.timezone=SystemV/CST6CDT -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] FAILURE 214s J1 | PeerSyncReplicationTest.test <<<
[junit4] > Throwable #1: java.lang.AssertionError: timeout waiting to see all nodes active
[junit4] > at __randomizedtesting.SeedInfo.seed([BB43F3AEC31A5791:3317CC746DE63A69]:0)
[junit4] > at org.apache.solr.cloud.PeerSyncReplicationTest.waitTillNodesActive(PeerSyncReplicationTest.java:326)
[junit4] > at org.apache.solr.cloud.PeerSyncReplicationTest.bringUpDeadNodeAndEnsureNoReplication(PeerSyncReplicationTest.java:277)
[junit4] > at org.apache.solr.cloud.PeerSyncReplicationTest.forceNodeFailureAndDoPeerSync(PeerSyncReplicationTest.java:259)
[junit4] > at org.apache.solr.cloud.PeerSyncReplicationTest.test(PeerSyncReplicationTest.java:138)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 1631902 INFO (SUITE-PeerSyncReplicationTest-seed#[BB43F3AEC31A5791]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_BB43F3AEC31A5791-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {other_tl1=PostingsFormat(name=Memory), range_facet_l_dv=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), rnd_s=PostingsFormat(name=Memory), multiDefault=FSTOrd50, a_t=FSTOrd50, intDefault=PostingsFormat(name=Memory), a_i1=FSTOrd50, id=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), range_facet_i_dv=FSTOrd50, text=PostingsFormat(name=Direct), range_facet_l=FSTOrd50, timestamp=FSTOrd50}, docValues:{range_facet_l_dv=DocValuesFormat(name=Lucene70), _version_=DocValuesFormat(name=Memory), range_facet_i_dv=DocValuesFormat(name=Lucene70), intDvoDefault=DocValuesFormat(name=Lucene70), timestamp=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=262, maxMBSortInHeap=7.29466106553652, sim=RandomSimilarity(queryNorm=true): {}, locale=zh-SG, timezone=SystemV/CST6CDT
[junit4] 2> NOTE: Mac OS X 10.11.6 x86_64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=3,threads=1,free=71227376,total=270876672
[junit4] 2> NOTE: All tests run in this JVM: [TestZkChroot, HdfsTlogReplayBufferedWhileIndexingTest, TestStressRecovery, ShowFileRequestHandlerTest, TestRecovery, AtomicUpdatesTest, SortByFunctionTest, RollingRestartTest, LukeRequestHandlerTest, TestDFRSimilarityFactory, TestSchemaVersionResource, PreAnalyzedFieldManagedSchemaCloudTest, LeaderInitiatedRecoveryOnCommitTest, TestHighFrequencyDictionaryFactory, DeleteStatusTest, TestFieldCacheSort, SuggesterWFSTTest, TestSSLRandomization, BooleanFieldTest, TestBackupRepositoryFactory, TestIndexSearcher, TestCloudInspectUtil, SliceStateTest, TestBinaryResponseWriter, DistributedQueryComponentCustomSortTest, TestSolrXml, TestUniqueKeyFieldResource, TestFuzzyAnalyzedSuggestions, AssignTest, CollectionsAPIAsyncDistributedZkTest, TestCrossCoreJoin, HdfsNNFailoverTest, UtilsToolTest, AutoCommitTest, BadIndexSchemaTest, TestDeleteCollectionOnDownNodes, TestSolrCoreProperties, TestPushWriter, CachingDirectoryFactoryTest, TestManagedResource, TestObjectReleaseTracker, BasicFunctionalityTest, SolrCloudExampleTest, TestPathTrie, HdfsBasicDistributedZkTest, NotRequiredUniqueKeyTest, TestSolrDeletionPolicy2, TestDocBasedVersionConstraints, SolrInfoMBeanTest, TestOmitPositions, LoggingHandlerTest, TestSolrCloudWithDelegationTokens, BasicAuthStandaloneTest, RuleEngineTest, TestCoreDiscovery, TestSystemIdResolver, TestSolrConfigHandler, SolrCoreMetricManagerTest, TestManagedSchemaAPI, OverseerModifyCollectionTest, TestDocSet, TestSolrCloudWithSecureImpersonation, TestHdfsCloudBackupRestore, TestConfigSetsAPIZkFailure, CollectionStateFormat2Test, TestSchemalessBufferedUpdates, TestImplicitCoreProperties, DebugComponentTest, TestRandomFlRTGCloud, HdfsSyncSliceTest, DistributedFacetExistsSmallTest, TestDefaultSearchFieldResource, TlogReplayBufferedWhileIndexingTest, DistributedSuggestComponentTest, BasicZkTest, TestStressInPlaceUpdates, ExplicitHLLTest, TestPointFields, DistanceUnitsTest, RequestHandlersTest, TestBulkSchemaConcurrent, NoCacheHeaderTest, TestManagedSynonymFilterFactory, URLClassifyProcessorTest, BaseCdcrDistributedZkTest, TestAnalyzeInfixSuggestions, TestRequestForwarding, DistributedFacetPivotLongTailTest, CustomCollectionTest, HdfsRecoveryZkTest, TestHashQParserPlugin, TestSimpleQParserPlugin, TestDynamicFieldResource, TestUseDocValuesAsStored2, TestDelegationWithHadoopAuth, TestDFISimilarityFactory, ResourceLoaderTest, MetricsHandlerTest, TemplateUpdateProcessorTest, CloneFieldUpdateProcessorFactoryTest, ShufflingReplicaListTransformerTest, TestScoreJoinQPNoScore, TestRealTimeGet, TestConfigsApi, TestConfigSetImmutable, DistributedFacetPivotSmallTest, TestRebalanceLeaders, TestJsonRequest, TestConfigSets, MergeStrategyTest, TestJsonFacetRefinement, TestMacroExpander, SolrCmdDistributorTest, TestRangeQuery, TestFoldingMultitermQuery, TestDownShardTolerantSearch, DistribDocExpirationUpdateProcessorTest, CurrencyFieldOpenExchangeTest, SimpleCollectionCreateDeleteTest, VMParamsZkACLAndCredentialsProvidersTest, TestSolrQueryParserDefaultOperatorResource, TestStressUserVersions, DefaultValueUpdateProcessorTest, ZkStateWriterTest, HdfsDirectoryFactoryTest, TestInitParams, UpdateRequestProcessorFactoryTest, TestFiltering, DateRangeFieldTest, DistributedTermsComponentTest, IndexSchemaTest, ClusterStateTest, TestUtils, OverriddenZkACLAndCredentialsProvidersTest, TestCustomSort, TestFieldCacheReopen, TestMiniSolrCloudCluster, TestTrie, LeaderFailoverAfterPartitionTest, TestFilteredDocIdSet, TestSubQueryTransformerCrossCore, TestJmxMonitoredMap, EnumFieldTest, TestConfigSetsAPI, SaslZkACLProviderTest, CdcrUpdateLogTest, TestLegacyFieldCache, TestSearchPerf, TestOverriddenPrefixQueryForCustomFieldType, TestQueryUtils, TestSolrCloudWithHadoopAuthPlugin, PeerSyncReplicationTest]
[junit4] Completed [265/693 (1!)] on J1 in 213.67s, 1 test, 1 failure <<< FAILURES!
[...truncated 63852 lines...]
[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3816
- Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3816/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC
1 tests failed.
FAILED: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test
Error Message:
Could not find collection:collection2
Stack Trace:
java.lang.AssertionError: Could not find collection:collection2
at __randomizedtesting.SeedInfo.seed([4E9ECDF09D230F36:C6CAF22A33DF62CE]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:159)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:144)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:139)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:856)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexingBatchPerRequestWithHttpSolrClient(FullSolrCloudDistribCmdsTest.java:620)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test(FullSolrCloudDistribCmdsTest.java:152)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11774 lines...]
[junit4] Suite: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
[junit4] 2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/init-core-data-001
[junit4] 2> 1918506 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[4E9ECDF09D230F36]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields
[junit4] 2> 1918515 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[4E9ECDF09D230F36]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776) w/ MAC_OS_X supressed clientAuth
[junit4] 2> 1918515 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[4E9ECDF09D230F36]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_z/
[junit4] 2> 1918517 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1918518 INFO (Thread-2235) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1918518 INFO (Thread-2235) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1918624 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.ZkTestServer start zk server on port:51000
[junit4] 2> 1918665 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1918700 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 1918765 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1918778 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1918811 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1918829 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1918845 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1918871 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1918885 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1918901 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1918918 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1921178 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/control-001/cores/collection1
[junit4] 2> 1921180 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1921182 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@657cc8a8{/_z,null,AVAILABLE}
[junit4] 2> 1921183 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@c800926{HTTP/1.1,[http/1.1]}{127.0.0.1:51009}
[junit4] 2> 1921183 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.e.j.s.Server Started @1926474ms
[junit4] 2> 1921183 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/tempDir-001/control/data, hostContext=/_z, hostPort=51009, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/control-001/cores}
[junit4] 2> 1921184 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1921184 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 1921184 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1921184 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1921184 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-06T14:11:18.929Z
[junit4] 2> 1921189 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1921189 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/control-001/solr.xml
[junit4] 2> 1921201 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51000/solr
[junit4] 2> 1921250 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [n:127.0.0.1:51009__z ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51009__z
[junit4] 2> 1921252 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [n:127.0.0.1:51009__z ] o.a.s.c.Overseer Overseer (id=97412073145434116-127.0.0.1:51009__z-n_0000000000) starting
[junit4] 2> 1921268 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [n:127.0.0.1:51009__z ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51009__z
[junit4] 2> 1921273 INFO (zkCallback-2567-thread-1-processing-n:127.0.0.1:51009__z) [n:127.0.0.1:51009__z ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1921392 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [n:127.0.0.1:51009__z ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/control-001/cores
[junit4] 2> 1921392 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [n:127.0.0.1:51009__z ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1921401 INFO (OverseerStateUpdate-97412073145434116-127.0.0.1:51009__z-n_0000000000) [n:127.0.0.1:51009__z ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 1922430 WARN (coreLoadExecutor-5737-thread-1-processing-n:127.0.0.1:51009__z) [n:127.0.0.1:51009__z c:control_collection x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 1922432 INFO (coreLoadExecutor-5737-thread-1-processing-n:127.0.0.1:51009__z) [n:127.0.0.1:51009__z c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1922444 INFO (coreLoadExecutor-5737-thread-1-processing-n:127.0.0.1:51009__z) [n:127.0.0.1:51009__z c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1922503 WARN (coreLoadExecutor-5737-thread-1-processing-n:127.0.0.1:51009__z) [n:127.0.0.1:51009__z c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1922505 INFO (coreLoadExecutor-5737-thread-1-processing-n:127.0.0.1:51009__z) [n:127.0.0.1:51009__z c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1922540 INFO (coreLoadExecutor-5737-thread-1-processing-n:127.0.0.1:51009__z) [n:127.0.0.1:51009__z c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 1922540 INFO (coreLoadExecutor-5737-thread-1-processing-n:127.0.0.1:51009__z) [n:127.0.0.1:51009__z c:control_collection x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/control-001/cores/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/control-001/cores/collection1/data/]
[junit4] 2> 1922540 INFO (coreLoadExecutor-5737-thread-1-processing-n:127.0.0.1:51009__z) [n:127.0.0.1:51009__z c:control_collection x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@24ae803a
[junit4] 2> 1922541 INFO (coreLoadExecutor-5737-thread-1-processing-n:127.0.0.1:51009__z) [n:127.0.0.1:51009__z c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=48, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1922546 WARN (coreLoadExecutor-5737-thread-1-processing-n:127.0.0.1:51009__z) [n:127.0.0.1:51009__z c:control_collection x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1922608 INFO (coreLoadExecutor-5737-thread-1-processing-n:127.0.0.1:51009__z) [n:127.0.0.1:51009__z c:control_collection x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1922608 INFO (coreLoadExecutor-5737-thread-1-processing-n:127.0.0.1:51009__z) [n:127.0.0.1:51009__z c:control_collection x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1922609 INFO (coreLoadExecutor-5737-thread-1-processing-n:127.0.0.1:51009__z) [n:127.0.0.1:51009__z c:control_collection x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1922609 INFO (coreLoadExecutor-5737-thread-1-processing-n:127.0.0.1:51009__z) [n:127.0.0.1:51009__z c:control_collection x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1922609 INFO (coreLoadExecutor-5737-thread-1-processing-n:127.0.0.1:51009__z) [n:127.0.0.1:51009__z c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=14, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1922610 INFO (coreLoadExecutor-5737-thread-1-processing-n:127.0.0.1:51009__z) [n:127.0.0.1:51009__z c:control_collection x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1e3af3c7[collection1] main]
[junit4] 2> 1922613 INFO (coreLoadExecutor-5737-thread-1-processing-n:127.0.0.1:51009__z) [n:127.0.0.1:51009__z c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1922613 INFO (coreLoadExecutor-5737-thread-1-processing-n:127.0.0.1:51009__z) [n:127.0.0.1:51009__z c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1922614 INFO (coreLoadExecutor-5737-thread-1-processing-n:127.0.0.1:51009__z) [n:127.0.0.1:51009__z c:control_collection x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1922615 INFO (searcherExecutor-5738-thread-1-processing-n:127.0.0.1:51009__z x:collection1 c:control_collection) [n:127.0.0.1:51009__z c:control_collection x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1e3af3c7[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1922615 INFO (coreLoadExecutor-5737-thread-1-processing-n:127.0.0.1:51009__z) [n:127.0.0.1:51009__z c:control_collection x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558593174618767360
[junit4] 2> 1922629 INFO (coreZkRegister-5730-thread-1-processing-n:127.0.0.1:51009__z x:collection1 c:control_collection) [n:127.0.0.1:51009__z c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1922629 INFO (coreZkRegister-5730-thread-1-processing-n:127.0.0.1:51009__z x:collection1 c:control_collection) [n:127.0.0.1:51009__z c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1922629 INFO (coreZkRegister-5730-thread-1-processing-n:127.0.0.1:51009__z x:collection1 c:control_collection) [n:127.0.0.1:51009__z c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:51009/_z/collection1/
[junit4] 2> 1922629 INFO (coreZkRegister-5730-thread-1-processing-n:127.0.0.1:51009__z x:collection1 c:control_collection) [n:127.0.0.1:51009__z c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1922630 INFO (coreZkRegister-5730-thread-1-processing-n:127.0.0.1:51009__z x:collection1 c:control_collection) [n:127.0.0.1:51009__z c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:51009/_z/collection1/ has no replicas
[junit4] 2> 1922637 INFO (coreZkRegister-5730-thread-1-processing-n:127.0.0.1:51009__z x:collection1 c:control_collection) [n:127.0.0.1:51009__z c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:51009/_z/collection1/ shard1
[junit4] 2> 1922809 INFO (coreZkRegister-5730-thread-1-processing-n:127.0.0.1:51009__z x:collection1 c:control_collection) [n:127.0.0.1:51009__z c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1923035 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1923038 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:51000/solr ready
[junit4] 2> 1923038 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 1923038 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
[junit4] 2> 1923383 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-1-001/cores/collection1
[junit4] 2> 1923385 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-1-001
[junit4] 2> 1923386 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1923389 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6439e64d{/_z,null,AVAILABLE}
[junit4] 2> 1923389 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@23da67ff{HTTP/1.1,[http/1.1]}{127.0.0.1:51015}
[junit4] 2> 1923389 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.e.j.s.Server Started @1928681ms
[junit4] 2> 1923390 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/_z, hostPort=51015, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-1-001/cores}
[junit4] 2> 1923390 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1923391 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 1923391 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1923391 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1923391 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-06T14:11:21.136Z
[junit4] 2> 1923398 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1923398 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-1-001/solr.xml
[junit4] 2> 1923412 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51000/solr
[junit4] 2> 1923434 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [n:127.0.0.1:51015__z ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1923445 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [n:127.0.0.1:51015__z ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51015__z
[junit4] 2> 1923450 INFO (zkCallback-2571-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1923451 INFO (zkCallback-2567-thread-3-processing-n:127.0.0.1:51009__z) [n:127.0.0.1:51009__z ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1923452 INFO (zkCallback-2577-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1923535 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [n:127.0.0.1:51015__z ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-1-001/cores
[junit4] 2> 1923535 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [n:127.0.0.1:51015__z ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1923539 INFO (OverseerStateUpdate-97412073145434116-127.0.0.1:51009__z-n_0000000000) [n:127.0.0.1:51009__z ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 1923646 INFO (zkCallback-2577-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1924567 WARN (coreLoadExecutor-5748-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 1924568 INFO (coreLoadExecutor-5748-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1924583 INFO (coreLoadExecutor-5748-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1924661 WARN (coreLoadExecutor-5748-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1924662 INFO (coreLoadExecutor-5748-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1924696 INFO (coreLoadExecutor-5748-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1924697 INFO (coreLoadExecutor-5748-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-1-001/cores/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-1-001/cores/collection1/data/]
[junit4] 2> 1924697 INFO (coreLoadExecutor-5748-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@24ae803a
[junit4] 2> 1924699 INFO (coreLoadExecutor-5748-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=48, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1924704 WARN (coreLoadExecutor-5748-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1924804 INFO (coreLoadExecutor-5748-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1924804 INFO (coreLoadExecutor-5748-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1924805 INFO (coreLoadExecutor-5748-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1924805 INFO (coreLoadExecutor-5748-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1924809 INFO (coreLoadExecutor-5748-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=14, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1924810 INFO (coreLoadExecutor-5748-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5738ebb3[collection1] main]
[junit4] 2> 1924812 INFO (coreLoadExecutor-5748-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1924813 INFO (coreLoadExecutor-5748-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1924813 INFO (coreLoadExecutor-5748-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1924814 INFO (searcherExecutor-5749-thread-1-processing-n:127.0.0.1:51015__z x:collection1 c:collection1) [n:127.0.0.1:51015__z c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5738ebb3[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1924815 INFO (coreLoadExecutor-5748-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558593176925634560
[junit4] 2> 1924829 INFO (coreZkRegister-5743-thread-1-processing-n:127.0.0.1:51015__z x:collection1 c:collection1) [n:127.0.0.1:51015__z c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1924829 INFO (coreZkRegister-5743-thread-1-processing-n:127.0.0.1:51015__z x:collection1 c:collection1) [n:127.0.0.1:51015__z c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1924829 INFO (coreZkRegister-5743-thread-1-processing-n:127.0.0.1:51015__z x:collection1 c:collection1) [n:127.0.0.1:51015__z c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:51015/_z/collection1/
[junit4] 2> 1924829 INFO (coreZkRegister-5743-thread-1-processing-n:127.0.0.1:51015__z x:collection1 c:collection1) [n:127.0.0.1:51015__z c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1924829 INFO (coreZkRegister-5743-thread-1-processing-n:127.0.0.1:51015__z x:collection1 c:collection1) [n:127.0.0.1:51015__z c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:51015/_z/collection1/ has no replicas
[junit4] 2> 1924839 INFO (coreZkRegister-5743-thread-1-processing-n:127.0.0.1:51015__z x:collection1 c:collection1) [n:127.0.0.1:51015__z c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:51015/_z/collection1/ shard2
[junit4] 2> 1924956 INFO (zkCallback-2577-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1925009 INFO (coreZkRegister-5743-thread-1-processing-n:127.0.0.1:51015__z x:collection1 c:collection1) [n:127.0.0.1:51015__z c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1925132 INFO (zkCallback-2577-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1925542 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-2-001/cores/collection1
[junit4] 2> 1925544 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-2-001
[junit4] 2> 1925545 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1925547 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2aebeef5{/_z,null,AVAILABLE}
[junit4] 2> 1925547 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@66ae5919{HTTP/1.1,[http/1.1]}{127.0.0.1:51019}
[junit4] 2> 1925547 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.e.j.s.Server Started @1930838ms
[junit4] 2> 1925547 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/_z, hostPort=51019, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-2-001/cores}
[junit4] 2> 1925548 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1925549 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 1925549 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1925549 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1925549 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-06T14:11:23.294Z
[junit4] 2> 1925554 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1925554 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-2-001/solr.xml
[junit4] 2> 1925581 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51000/solr
[junit4] 2> 1925601 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [n:127.0.0.1:51019__z ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1925611 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [n:127.0.0.1:51019__z ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51019__z
[junit4] 2> 1925614 INFO (zkCallback-2571-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1925614 INFO (zkCallback-2567-thread-1-processing-n:127.0.0.1:51009__z) [n:127.0.0.1:51009__z ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1925615 INFO (zkCallback-2577-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1925615 INFO (zkCallback-2583-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1925725 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [n:127.0.0.1:51019__z ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-2-001/cores
[junit4] 2> 1925725 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [n:127.0.0.1:51019__z ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1925727 INFO (zkCallback-2577-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1925733 INFO (OverseerStateUpdate-97412073145434116-127.0.0.1:51009__z-n_0000000000) [n:127.0.0.1:51009__z ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard3
[junit4] 2> 1925847 INFO (zkCallback-2577-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1925847 INFO (zkCallback-2583-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1926753 WARN (coreLoadExecutor-5759-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 1926754 INFO (coreLoadExecutor-5759-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1926767 INFO (coreLoadExecutor-5759-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1926829 WARN (coreLoadExecutor-5759-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1926831 INFO (coreLoadExecutor-5759-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1926872 INFO (coreLoadExecutor-5759-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1926873 INFO (coreLoadExecutor-5759-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-2-001/cores/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-2-001/cores/collection1/data/]
[junit4] 2> 1926873 INFO (coreLoadExecutor-5759-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@24ae803a
[junit4] 2> 1926874 INFO (coreLoadExecutor-5759-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=48, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1926879 WARN (coreLoadExecutor-5759-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1926967 INFO (coreLoadExecutor-5759-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1926967 INFO (coreLoadExecutor-5759-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1926969 INFO (coreLoadExecutor-5759-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1926969 INFO (coreLoadExecutor-5759-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1926973 INFO (coreLoadExecutor-5759-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=14, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1926973 INFO (coreLoadExecutor-5759-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@56e3bf1a[collection1] main]
[junit4] 2> 1926975 INFO (coreLoadExecutor-5759-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1926976 INFO (coreLoadExecutor-5759-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1926976 INFO (coreLoadExecutor-5759-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1926977 INFO (searcherExecutor-5760-thread-1-processing-n:127.0.0.1:51019__z x:collection1 c:collection1) [n:127.0.0.1:51019__z c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@56e3bf1a[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1926977 INFO (coreLoadExecutor-5759-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558593179192655872
[junit4] 2> 1926999 INFO (coreZkRegister-5754-thread-1-processing-n:127.0.0.1:51019__z x:collection1 c:collection1) [n:127.0.0.1:51019__z c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1926999 INFO (coreZkRegister-5754-thread-1-processing-n:127.0.0.1:51019__z x:collection1 c:collection1) [n:127.0.0.1:51019__z c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1926999 INFO (coreZkRegister-5754-thread-1-processing-n:127.0.0.1:51019__z x:collection1 c:collection1) [n:127.0.0.1:51019__z c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:51019/_z/collection1/
[junit4] 2> 1926999 INFO (coreZkRegister-5754-thread-1-processing-n:127.0.0.1:51019__z x:collection1 c:collection1) [n:127.0.0.1:51019__z c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1926999 INFO (coreZkRegister-5754-thread-1-processing-n:127.0.0.1:51019__z x:collection1 c:collection1) [n:127.0.0.1:51019__z c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:51019/_z/collection1/ has no replicas
[junit4] 2> 1927008 INFO (coreZkRegister-5754-thread-1-processing-n:127.0.0.1:51019__z x:collection1 c:collection1) [n:127.0.0.1:51019__z c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:51019/_z/collection1/ shard3
[junit4] 2> 1927124 INFO (zkCallback-2583-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1927124 INFO (zkCallback-2577-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1927183 INFO (coreZkRegister-5754-thread-1-processing-n:127.0.0.1:51019__z x:collection1 c:collection1) [n:127.0.0.1:51019__z c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1927301 INFO (zkCallback-2577-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1927301 INFO (zkCallback-2583-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1927868 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-3-001/cores/collection1
[junit4] 2> 1927869 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-3-001
[junit4] 2> 1927870 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1927872 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6791b7b8{/_z,null,AVAILABLE}
[junit4] 2> 1927872 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2896d3ce{HTTP/1.1,[http/1.1]}{127.0.0.1:51023}
[junit4] 2> 1927872 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.e.j.s.Server Started @1933163ms
[junit4] 2> 1927872 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/tempDir-001/jetty3, solrconfig=solrconfig.xml, hostContext=/_z, hostPort=51023, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-3-001/cores}
[junit4] 2> 1927873 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1927873 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 1927873 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1927873 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1927873 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-06T14:11:25.618Z
[junit4] 2> 1927877 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1927877 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-3-001/solr.xml
[junit4] 2> 1927896 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51000/solr
[junit4] 2> 1927916 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [n:127.0.0.1:51023__z ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1927924 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [n:127.0.0.1:51023__z ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51023__z
[junit4] 2> 1927928 INFO (zkCallback-2577-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1927928 INFO (zkCallback-2583-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1927930 INFO (zkCallback-2589-thread-1-processing-n:127.0.0.1:51023__z) [n:127.0.0.1:51023__z ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1927930 INFO (zkCallback-2571-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1927930 INFO (zkCallback-2567-thread-3-processing-n:127.0.0.1:51009__z) [n:127.0.0.1:51009__z ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1928042 INFO (zkCallback-2577-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1928043 INFO (zkCallback-2583-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1928078 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [n:127.0.0.1:51023__z ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-3-001/cores
[junit4] 2> 1928078 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [n:127.0.0.1:51023__z ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1928084 INFO (OverseerStateUpdate-97412073145434116-127.0.0.1:51009__z-n_0000000000) [n:127.0.0.1:51009__z ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 1928211 INFO (zkCallback-2589-thread-1-processing-n:127.0.0.1:51023__z) [n:127.0.0.1:51023__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1928211 INFO (zkCallback-2577-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1928212 INFO (zkCallback-2583-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1929112 WARN (coreLoadExecutor-5770-thread-1-processing-n:127.0.0.1:51023__z) [n:127.0.0.1:51023__z c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 1929113 INFO (coreLoadExecutor-5770-thread-1-processing-n:127.0.0.1:51023__z) [n:127.0.0.1:51023__z c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1929124 INFO (coreLoadExecutor-5770-thread-1-processing-n:127.0.0.1:51023__z) [n:127.0.0.1:51023__z c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1929196 WARN (coreLoadExecutor-5770-thread-1-processing-n:127.0.0.1:51023__z) [n:127.0.0.1:51023__z c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1929197 INFO (coreLoadExecutor-5770-thread-1-processing-n:127.0.0.1:51023__z) [n:127.0.0.1:51023__z c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1929234 INFO (coreLoadExecutor-5770-thread-1-processing-n:127.0.0.1:51023__z) [n:127.0.0.1:51023__z c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1929234 INFO (coreLoadExecutor-5770-thread-1-processing-n:127.0.0.1:51023__z) [n:127.0.0.1:51023__z c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-3-001/cores/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-3-001/cores/collection1/data/]
[junit4] 2> 1929235 INFO (coreLoadExecutor-5770-thread-1-processing-n:127.0.0.1:51023__z) [n:127.0.0.1:51023__z c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@24ae803a
[junit4] 2> 1929236 INFO (coreLoadExecutor-5770-thread-1-processing-n:127.0.0.1:51023__z) [n:127.0.0.1:51023__z c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=48, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1929241 WARN (coreLoadExecutor-5770-thread-1-processing-n:127.0.0.1:51023__z) [n:127.0.0.1:51023__z c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1929312 INFO (coreLoadExecutor-5770-thread-1-processing-n:127.0.0.1:51023__z) [n:127.0.0.1:51023__z c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1929312 INFO (coreLoadExecutor-5770-thread-1-processing-n:127.0.0.1:51023__z) [n:127.0.0.1:51023__z c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1929314 INFO (coreLoadExecutor-5770-thread-1-processing-n:127.0.0.1:51023__z) [n:127.0.0.1:51023__z c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1929314 INFO (coreLoadExecutor-5770-thread-1-processing-n:127.0.0.1:51023__z) [n:127.0.0.1:51023__z c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1929315 INFO (coreLoadExecutor-5770-thread-1-processing-n:127.0.0.1:51023__z) [n:127.0.0.1:51023__z c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=14, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1929316 INFO (coreLoadExecutor-5770-thread-1-processing-n:127.0.0.1:51023__z) [n:127.0.0.1:51023__z c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@18902755[collection1] main]
[junit4] 2> 1929319 INFO (coreLoadExecutor-5770-thread-1-processing-n:127.0.0.1:51023__z) [n:127.0.0.1:51023__z c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1929319 INFO (coreLoadExecutor-5770-thread-1-processing-n:127.0.0.1:51023__z) [n:127.0.0.1:51023__z c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1929319 INFO (coreLoadExecutor-5770-thread-1-processing-n:127.0.0.1:51023__z) [n:127.0.0.1:51023__z c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1929321 INFO (searcherExecutor-5771-thread-1-processing-n:127.0.0.1:51023__z x:collection1 c:collection1) [n:127.0.0.1:51023__z c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@18902755[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1929322 INFO (coreLoadExecutor-5770-thread-1-processing-n:127.0.0.1:51023__z) [n:127.0.0.1:51023__z c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558593181651566592
[junit4] 2> 1929333 INFO (coreZkRegister-5765-thread-1-processing-n:127.0.0.1:51023__z x:collection1 c:collection1) [n:127.0.0.1:51023__z c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1929334 INFO (coreZkRegister-5765-thread-1-processing-n:127.0.0.1:51023__z x:collection1 c:collection1) [n:127.0.0.1:51023__z c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1929334 INFO (coreZkRegister-5765-thread-1-processing-n:127.0.0.1:51023__z x:collection1 c:collection1) [n:127.0.0.1:51023__z c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:51023/_z/collection1/
[junit4] 2> 1929334 INFO (coreZkRegister-5765-thread-1-processing-n:127.0.0.1:51023__z x:collection1 c:collection1) [n:127.0.0.1:51023__z c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1929334 INFO (coreZkRegister-5765-thread-1-processing-n:127.0.0.1:51023__z x:collection1 c:collection1) [n:127.0.0.1:51023__z c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:51023/_z/collection1/ has no replicas
[junit4] 2> 1929340 INFO (coreZkRegister-5765-thread-1-processing-n:127.0.0.1:51023__z x:collection1 c:collection1) [n:127.0.0.1:51023__z c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:51023/_z/collection1/ shard1
[junit4] 2> 1929457 INFO (zkCallback-2583-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1929457 INFO (zkCallback-2589-thread-1-processing-n:127.0.0.1:51023__z) [n:127.0.0.1:51023__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1929457 INFO (zkCallback-2577-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1929513 INFO (coreZkRegister-5765-thread-1-processing-n:127.0.0.1:51023__z x:collection1 c:collection1) [n:127.0.0.1:51023__z c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1929624 INFO (zkCallback-2577-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1929624 INFO (zkCallback-2589-thread-1-processing-n:127.0.0.1:51023__z) [n:127.0.0.1:51023__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1929624 INFO (zkCallback-2583-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1930841 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-4-001/cores/collection1
[junit4] 2> 1930844 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 4 in directory /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-4-001
[junit4] 2> 1930844 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1930846 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@675d65d2{/_z,null,AVAILABLE}
[junit4] 2> 1930847 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@79fea500{HTTP/1.1,[http/1.1]}{127.0.0.1:51032}
[junit4] 2> 1930847 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.e.j.s.Server Started @1936138ms
[junit4] 2> 1930847 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/tempDir-001/jetty4, solrconfig=solrconfig.xml, hostContext=/_z, hostPort=51032, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-4-001/cores}
[junit4] 2> 1930848 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1930849 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 1930849 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1930849 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1930849 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-06T14:11:28.594Z
[junit4] 2> 1930853 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1930854 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-4-001/solr.xml
[junit4] 2> 1930879 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51000/solr
[junit4] 2> 1930899 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [n:127.0.0.1:51032__z ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1930911 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [n:127.0.0.1:51032__z ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51032__z
[junit4] 2> 1930914 INFO (zkCallback-2583-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1930914 INFO (zkCallback-2577-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1930915 INFO (zkCallback-2571-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1930915 INFO (zkCallback-2567-thread-1-processing-n:127.0.0.1:51009__z) [n:127.0.0.1:51009__z ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1930915 INFO (zkCallback-2589-thread-1-processing-n:127.0.0.1:51023__z) [n:127.0.0.1:51023__z ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1930916 INFO (zkCallback-2595-thread-1-processing-n:127.0.0.1:51032__z) [n:127.0.0.1:51032__z ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1931037 INFO (zkCallback-2577-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1931037 INFO (zkCallback-2589-thread-1-processing-n:127.0.0.1:51023__z) [n:127.0.0.1:51023__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1931037 INFO (zkCallback-2583-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1931095 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [n:127.0.0.1:51032__z ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-4-001/cores
[junit4] 2> 1931095 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [n:127.0.0.1:51032__z ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1931099 INFO (OverseerStateUpdate-97412073145434116-127.0.0.1:51009__z-n_0000000000) [n:127.0.0.1:51009__z ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 1931215 INFO (zkCallback-2595-thread-1-processing-n:127.0.0.1:51032__z) [n:127.0.0.1:51032__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1931215 INFO (zkCallback-2577-thread-1-processing-n:127.0.0.1:51015__z) [n:127.0.0.1:51015__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1931215 INFO (zkCallback-2583-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1931216 INFO (zkCallback-2589-thread-1-processing-n:127.0.0.1:51023__z) [n:127.0.0.1:51023__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1932138 WARN (coreLoadExecutor-5781-thread-1-processing-n:127.0.0.1:51032__z) [n:127.0.0.1:51032__z c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 1932139 INFO (coreLoadExecutor-5781-thread-1-processing-n:127.0.0.1:51032__z) [n:127.0.0.1:51032__z c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1932152 INFO (coreLoadExecutor-5781-thread-1-processing-n:127.0.0.1:51032__z) [n:127.0.0.1:51032__z c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1932213 WARN (coreLoadExecutor-5781-thread-1-processing-n:127.0.0.1:51032__z) [n:127.0.0.1:51032__z c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1932214 INFO (coreLoadExecutor-5781-thread-1-processing-n:127.0.0.1:51032__z) [n:127.0.0.1:51032__z c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1932249 INFO (coreLoadExecutor-5781-thread-1-processing-n:127.0.0.1:51032__z) [n:127.0.0.1:51032__z c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1932249 INFO (coreLoadExecutor-5781-thread-1-processing-n:127.0.0.1:51032__z) [n:127.0.0.1:51032__z c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-4-001/cores/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001/shard-4-001/cores/collection1/data/]
[junit4] 2> 1932249 INFO (coreLoadExecutor-5781-thread-1-processing-n:127.0.0.1:51032__z) [n:127.0.0.1:51032__z c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@24ae803a
[junit4] 2> 1932250 INFO (coreLoadExecutor-5781-thread-1-processing-n:127.0.0.1:51032__z) [n:127.0.0.1:51032__z c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=48, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1932257 WARN (coreLoadExecutor-5781-thread-1-processing-n:127.0.0.1:51032__z) [n:127.0.0.1:51032__z c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1932357 INFO (coreLoadExecutor-5781-thread-1-processing-n:127.0.0.1:51032__z) [n:127.0.0.1:51032__z c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1932357 INFO (coreLoadExecutor-5781-thread-1-processing-n:127.0.0.1:51032__z) [n:127.0.0.1:51032__z c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1932359 INFO (coreLoadExecutor-5781-thread-1-processing-n:127.0.0.1:51032__z) [n:127.0.0.1:51032__z c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1932359 INFO (coreLoadExecutor-5781-thread-1-processing-n:127.0.0.1:51032__z) [n:127.0.0.1:51032__z c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1932360 INFO (coreLoadExecutor-5781-thread-1-processing-n:127.0.0.1:51032__z) [n:127.0.0.1:51032__z c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=14, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1932361 INFO (coreLoadExecutor-5781-thread-1-processing-n:127.0.0.1:51032__z) [n:127.0.0.1:51032__z c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@36523f5[collection1] main]
[junit4] 2> 1932363 INFO (coreLoadExecutor-5781-thread-1-processing-n:127.0.0.1:51032__z) [n:127.0.0.1:51032__z c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1932364 INFO (coreLoadExecutor-5781-thread-1-processing-n:127.0.0.1:51032__z) [n:127.0.0.1:51032__z c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1932364 INFO (coreLoadExecutor-5781-thread-1-processing-n:127.0.0.1:51032__z) [n:127.0.0.1:51032__z c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1932367 INFO (searcherExecutor-5782-thread-1-processing-n:127.0.0.1:51032__z x:collection1 c:collection1) [n:127.0.0.1:51032__z c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@36523f5[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1932367 INFO (coreLoadExecutor-5781-thread-1-processing-n:127.0.0.1:51032__z) [n:127.0.0.1:51032__z c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558593184844480512
[junit4] 2> 1932373 INFO (coreZkRegister-5776-thread-1-processing-n:127.0.0.1:51032__z x:collection1 c:collection1) [n:127.0.0.1:51032__z c:collection1 s:shard2 r:core_node4 x:collection1] o.a.s.c.ZkController Core needs to recover:collection1
[junit4] 2> 1932374 INFO (updateExecutor-2592-thread-1-processing-n:127.0.0.1:51032__z x:collection1 s:shard2 c:collection1 r:core_node4) [n:127.0.0.1:51032__z c:collection1 s:shard2 r:core_node4 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 1932374 INFO (recoveryExecutor-2593-thread-1-processing-n:127.0.0.1:51032__z x:collection1 s:shard2 c:collection1 r:core_node4) [n:127.0.0.1:51032__z c:collection1 s:shard2 r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 1932375 INFO (recoveryExecutor-2593-thread-1-processing-n:127.0.0.1:51032__z x:collection1 s:shard2 c:collection1 r:core_node4) [n:127.0.0.1:51032__z c:collection1 s:shard2 r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 1932375 INFO (recoveryExecutor-2593-thread-1-processing-n:127.0.0.1:51032__z x:collection1 s:shard2 c:collection1 r:core_node4) [n:127.0.0.1:51032__z c:collection1 s:shard2 r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1]
[junit4] 2> 1932375 INFO (recoveryExecutor-2593-thread-1-processing-n:127.0.0.1:51032__z x:collection1 s:shard2 c:collection1 r:core_node4) [n:127.0.0.1:51032__z c:collection1 s:shard2 r:core_node4 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 1932376 INFO (recoveryExecutor-2593-thread-1-processing-n:127.0.0.1:51032__z x:collection1 s:shard2 c:collection1 r:core_node4) [n:127.0.0.1:51032__z c:collection1 s:shard2 r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core [collection1] as recovering, leader is [http://127.0.0.1:51015/_z/collection1/] and I am [http://127.0.0.1:51032/_z/collection1/]
[junit4] 2> 1932381 INFO (recoveryExecutor-2593-thread-1-processing-n:127.0.0.1:51032__z x:collection1 s:shard2 c:collection1 r:core_node4) [n:127.0.0.1:51032__z c:collection1 s:shard2 r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:51015/_z]; [WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:51032__z&coreNodeName=core_node4&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 1932385 INFO (qtp54979201-13497) [n:127.0.0.1:51015__z ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 1932385 INFO (qtp54979201-13497) [n:127.0.0.1:51015__z ] o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 (shard2 of collection1) have state: recovering
[junit4] 2> 1932385 INFO (qtp54979201-13497) [n:127.0.0.1:51015__z ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard2, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:51032__z, coreNodeName=core_node4, onlyIfActiveCheckResult=false, nodeProps: core_node4:{"core":"collection1","base_url":"http://127.0.0.1:51032/_z","node_name":"127.0.0.1:51032__z","state":"down"}
[junit4] 2> 1932487 INFO (zkCallback-2595-thread-1-processing-n:127.0.0.1:51032__z) [n:127.0.0.1:51032__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1932487 INFO (zkCallback-2583-thread-1-processing-n:127.0.0.1:51019__z) [n:127.0.0.1:51019__z ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1932487 INFO (zkCallback-2577-thread-1-processing-n:127
[...truncated too long message...]
ECDF09D230F36]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1427976470
[junit4] 2> 1995156 INFO (zkCallback-2602-thread-2-processing-n:127.0.0.1:51037__z) [n:127.0.0.1:51037__z ] o.a.s.c.Overseer Overseer (id=97412073145434133-127.0.0.1:51037__z-n_0000000005) starting
[junit4] 2> 1995167 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.Overseer Overseer (id=97412073145434133-127.0.0.1:51037__z-n_0000000005) closing
[junit4] 2> 1995168 INFO (OverseerStateUpdate-97412073145434133-127.0.0.1:51037__z-n_0000000005) [n:127.0.0.1:51037__z ] o.a.s.c.Overseer According to ZK I (id=97412073145434133-127.0.0.1:51037__z-n_0000000005) am no longer a leader.
[junit4] 2> 1995168 INFO (OverseerStateUpdate-97412073145434133-127.0.0.1:51037__z-n_0000000005) [n:127.0.0.1:51037__z ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:51037__z
[junit4] 2> 1995168 INFO (OverseerCollectionConfigSetProcessor-97412073145434133-127.0.0.1:51037__z-n_0000000005) [n:127.0.0.1:51037__z ] o.a.s.c.OverseerTaskProcessor According to ZK I (id=97412073145434133-127.0.0.1:51037__z-n_0000000005) am no longer a leader.
[junit4] 2> 1995169 WARN (zkCallback-2602-thread-2-processing-n:127.0.0.1:51037__z) [n:127.0.0.1:51037__z ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 1995170 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 1995171 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@6776f543{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 1995171 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@7e3eb2f3{/_z,null,UNAVAILABLE}
[junit4] 2> 1995173 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.ChaosMonkey monkey: stop shard! 51048
[junit4] 2> 1995175 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[4E9ECDF09D230F36]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:51000 51000
[junit4] 2> 1995190 INFO (Thread-2235) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:51000 51000
[junit4] 2> 1998429 WARN (Thread-2235) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 14 /solr/aliases.json
[junit4] 2> 14 /solr/clusterprops.json
[junit4] 2> 10 /solr/configs/conf1
[junit4] 2> 7 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 14 /solr/clusterstate.json
[junit4] 2> 6 /solr/collections/collection1/state.json
[junit4] 2> 5 /solr/collections/collection2/state.json
[junit4] 2> 2 /solr/overseer_elect/election/97412073145434124-127.0.0.1:51019__z-n_0000000002
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 87 /solr/overseer/queue
[junit4] 2> 63 /solr/overseer/collection-queue-work
[junit4] 2> 17 /solr/overseer/queue-work
[junit4] 2> 14 /solr/live_nodes
[junit4] 2> 14 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=FullSolrCloudDistribCmdsTest -Dtests.method=test -Dtests.seed=4E9ECDF09D230F36 -Dtests.slow=true -Dtests.locale=und -Dtests.timezone=Pacific/Niue -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] FAILURE 80.0s J0 | FullSolrCloudDistribCmdsTest.test <<<
[junit4] > Throwable #1: java.lang.AssertionError: Could not find collection:collection2
[junit4] > at __randomizedtesting.SeedInfo.seed([4E9ECDF09D230F36:C6CAF22A33DF62CE]:0)
[junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:159)
[junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:144)
[junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:139)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:856)
[junit4] > at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexingBatchPerRequestWithHttpSolrClient(FullSolrCloudDistribCmdsTest.java:620)
[junit4] > at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test(FullSolrCloudDistribCmdsTest.java:152)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 1998468 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[4E9ECDF09D230F36]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_4E9ECDF09D230F36-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70), sim=RandomSimilarity(queryNorm=true): {}, locale=und, timezone=Pacific/Niue
[junit4] 2> NOTE: Mac OS X 10.11.6 x86_64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=3,threads=1,free=193839320,total=493879296
[junit4] 2> NOTE: All tests run in this JVM: [TestLFUCache, TestBulkSchemaAPI, FileUtilsTest, TestSweetSpotSimilarityFactory, TestConfigSetsAPIExclusivity, BitVectorTest, ShardRoutingTest, FullHLLTest, TestFieldSortValues, ReplicaListTransformerTest, TestAddFieldRealTimeGet, CdcrBootstrapTest, RulesTest, SampleTest, TestXmlQParserPlugin, AsyncCallRequestStatusResponseTest, ZkControllerTest, TestIntervalFaceting, TestManagedSchemaThreadSafety, TestPhraseSuggestions, TestFieldCacheSortRandom, SpatialFilterTest, StressHdfsTest, TestStressCloudBlindAtomicUpdates, SolrTestCaseJ4Test, CoreAdminRequestStatusTest, RequiredFieldsTest, QueryEqualityTest, SmileWriterTest, SolrRequestParserTest, SolrIndexConfigTest, TestRequestStatusCollectionAPI, TestSort, SolrMetricManagerTest, TestHashPartitioner, OutputWriterTest, TestCollectionAPI, TestDynamicFieldCollectionResource, TestCustomDocTransformer, TestSolrDeletionPolicy1, BufferStoreTest, CheckHdfsIndexTest, TestPostingsSolrHighlighter, TestMergePolicyConfig, TestWordDelimiterFilterFactory, SuggesterFSTTest, LeaderElectionContextKeyTest, TestElisionMultitermQuery, TestCSVLoader, TestTestInjection, TestConfigOverlay, TestSolrDynamicMBean, TestManagedSchema, TestMaxScoreQueryParser, TestBinaryField, TestCorePropertiesReload, StatelessScriptUpdateProcessorFactoryTest, AnalysisErrorHandlingTest, ForceLeaderTest, TestStandardQParsers, TestQueryWrapperFilter, TestSubQueryTransformerDistrib, OpenExchangeRatesOrgProviderTest, TestExtendedDismaxParser, BasicAuthIntegrationTest, DocumentAnalysisRequestHandlerTest, LeaderElectionTest, TestJavabinTupleStreamParser, OverseerTaskQueueTest, MultiTermTest, TestStressLiveNodes, TestNonDefinedSimilarityFactory, TestStressVersions, TestReplicaProperties, TestCollapseQParserPlugin, AnalyticsMergeStrategyTest, BigEndianAscendingWordDeserializerTest, TestSchemaManager, TestFunctionQuery, TestClassicSimilarityFactory, ManagedSchemaRoundRobinCloudTest, TestRTGBase, TestLazyCores, TestRestoreCore, TestConfigReload, CurrencyFieldXmlFileTest, BasicDistributedZkTest, TestUninvertingReader, UniqFieldsUpdateProcessorFactoryTest, TestScoreJoinQPScore, TestTrackingShardHandlerFactory, SyncSliceTest, TestRemoteStreaming, CollectionReloadTest, XsltUpdateRequestHandlerTest, MoreLikeThisHandlerTest, TriLevelCompositeIdRoutingTest, StatsReloadRaceTest, TermsComponentTest, SolrIndexMetricsTest, SynonymTokenizerTest, PrimUtilsTest, TestCloudManagedSchema, MissingSegmentRecoveryTest, TestJmxIntegration, HighlighterTest, IndexSchemaRuntimeFieldTest, TestBM25SimilarityFactory, SuggestComponentTest, TestCoreContainer, TestShardHandlerFactory, TestGraphTermsQParserPlugin, OverseerCollectionConfigSetProcessorTest, TestCloudPivotFacet, DistributedQueryComponentOptimizationTest, ClusterStateUpdateTest, QueryElevationComponentTest, TestExpandComponent, TestSolrCloudSnapshots, TestIBSimilarityFactory, PeerSyncTest, TestCopyFieldCollectionResource, PrimitiveFieldTypeTest, TestFieldCache, ZkCLITest, ZkStateReaderTest, PluginInfoTest, TestSchemaNameResource, RankQueryTest, TestPKIAuthenticationPlugin, BJQParserTest, CreateCollectionCleanupTest, TestOnReconnectListenerSupport, TestMacros, TestSlowCompositeReaderWrapper, TestGroupingSearch, DOMUtilTest, TestQuerySenderNoQuery, TestTrieFacet, DocExpirationUpdateProcessorFactoryTest, DistributedFacetPivotSmallAdvancedTest, DistribCursorPagingTest, GraphQueryTest, BadCopyFieldTest, ReplaceNodeTest, SOLR749Test, CopyFieldTest, LeaderInitiatedRecoveryOnShardRestartTest, TestSegmentSorting, HdfsRecoverLeaseTest, HdfsChaosMonkeyNothingIsSafeTest, SolrGraphiteReporterTest, NumericFieldsTest, ZkNodePropsTest, TestSizeLimitedDistributedMap, TestSQLHandlerNonCloud, UpdateLogTest, CoreSorterTest, DistribJoinFromCollectionTest, CleanupOldIndexTest, TestInitQParser, HdfsThreadLeakTest, TestXIncludeConfig, SolrMetricsIntegrationTest, IndexBasedSpellCheckerTest, DocumentBuilderTest, TestNRTOpen, TestLRUCache, TestRandomDVFaceting, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest, RecoveryZkTest, FullSolrCloudDistribCmdsTest]
[junit4] Completed [352/693 (1!)] on J0 in 79.98s, 1 test, 1 failure <<< FAILURES!
[...truncated 63603 lines...]
[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3815
- Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3815/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC
2 tests failed.
FAILED: org.apache.solr.cloud.HttpPartitionTest.test
Error Message:
Collection not found: collMinRf_1x3
Stack Trace:
org.apache.solr.common.SolrException: Collection not found: collMinRf_1x3
at __randomizedtesting.SeedInfo.seed([F2B8AEDAE2C0C2AE:7AEC91004C3CAF56]:0)
at org.apache.solr.client.solrj.impl.CloudSolrClient.getCollectionNames(CloudSolrClient.java:1376)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1072)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1042)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.HttpPartitionTest.sendDoc(HttpPartitionTest.java:611)
at org.apache.solr.cloud.HttpPartitionTest.testMinRf(HttpPartitionTest.java:210)
at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.CustomCollectionTest.testRouteFieldForImplicitRouter
Error Message:
Collection not found: withShardField
Stack Trace:
org.apache.solr.common.SolrException: Collection not found: withShardField
at __randomizedtesting.SeedInfo.seed([F2B8AEDAE2C0C2AE:A7E846484E390D5E]:0)
at org.apache.solr.client.solrj.impl.CloudSolrClient.getCollectionNames(CloudSolrClient.java:1376)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1072)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1042)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:232)
at org.apache.solr.cloud.CustomCollectionTest.testRouteFieldForImplicitRouter(CustomCollectionTest.java:141)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 12249 lines...]
[junit4] Suite: org.apache.solr.cloud.CustomCollectionTest
[junit4] 2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/init-core-data-001
[junit4] 2> 3016752 INFO (SUITE-CustomCollectionTest-seed#[F2B8AEDAE2C0C2AE]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields
[junit4] 2> 3016758 INFO (SUITE-CustomCollectionTest-seed#[F2B8AEDAE2C0C2AE]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN) w/ MAC_OS_X supressed clientAuth
[junit4] 2> 3016759 INFO (SUITE-CustomCollectionTest-seed#[F2B8AEDAE2C0C2AE]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/tempDir-001
[junit4] 2> 3016759 INFO (SUITE-CustomCollectionTest-seed#[F2B8AEDAE2C0C2AE]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 3016759 INFO (Thread-3332) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 3016759 INFO (Thread-3332) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 3016859 INFO (SUITE-CustomCollectionTest-seed#[F2B8AEDAE2C0C2AE]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:63040
[junit4] 2> 3016881 INFO (jetty-launcher-3541-thread-1) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 3016881 INFO (jetty-launcher-3541-thread-2) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 3016882 INFO (jetty-launcher-3541-thread-3) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 3016883 INFO (jetty-launcher-3541-thread-4) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 3016885 INFO (jetty-launcher-3541-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7665a33a{/solr,null,AVAILABLE}
[junit4] 2> 3016887 INFO (jetty-launcher-3541-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@728aea48{HTTP/1.1,[http/1.1]}{127.0.0.1:63042}
[junit4] 2> 3016887 INFO (jetty-launcher-3541-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@663a31ee{/solr,null,AVAILABLE}
[junit4] 2> 3016887 INFO (jetty-launcher-3541-thread-2) [ ] o.e.j.s.Server Started @3022414ms
[junit4] 2> 3016887 INFO (jetty-launcher-3541-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=63042}
[junit4] 2> 3016888 ERROR (jetty-launcher-3541-thread-2) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3016888 INFO (jetty-launcher-3541-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 3016888 INFO (jetty-launcher-3541-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3016888 INFO (jetty-launcher-3541-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3016888 INFO (jetty-launcher-3541-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-04T17:45:22.162Z
[junit4] 2> 3016888 INFO (jetty-launcher-3541-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3cb25bfa{/solr,null,AVAILABLE}
[junit4] 2> 3016889 INFO (jetty-launcher-3541-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@55d53c51{HTTP/1.1,[http/1.1]}{127.0.0.1:63044}
[junit4] 2> 3016889 INFO (jetty-launcher-3541-thread-3) [ ] o.e.j.s.Server Started @3022416ms
[junit4] 2> 3016889 INFO (jetty-launcher-3541-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=63044}
[junit4] 2> 3016889 ERROR (jetty-launcher-3541-thread-3) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3016889 INFO (jetty-launcher-3541-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 3016889 INFO (jetty-launcher-3541-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3016889 INFO (jetty-launcher-3541-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3016889 INFO (jetty-launcher-3541-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-04T17:45:22.163Z
[junit4] 2> 3016890 INFO (jetty-launcher-3541-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@19bf0f0d{/solr,null,AVAILABLE}
[junit4] 2> 3016890 INFO (jetty-launcher-3541-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@dd6db30{HTTP/1.1,[http/1.1]}{127.0.0.1:63043}
[junit4] 2> 3016890 INFO (jetty-launcher-3541-thread-4) [ ] o.e.j.s.Server Started @3022418ms
[junit4] 2> 3016891 INFO (jetty-launcher-3541-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=63043}
[junit4] 2> 3016891 ERROR (jetty-launcher-3541-thread-4) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3016891 INFO (jetty-launcher-3541-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 3016892 INFO (jetty-launcher-3541-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2f5c770f{HTTP/1.1,[http/1.1]}{127.0.0.1:63046}
[junit4] 2> 3016892 INFO (jetty-launcher-3541-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3016892 INFO (jetty-launcher-3541-thread-1) [ ] o.e.j.s.Server Started @3022419ms
[junit4] 2> 3016892 INFO (jetty-launcher-3541-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3016892 INFO (jetty-launcher-3541-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=63046}
[junit4] 2> 3016892 INFO (jetty-launcher-3541-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-04T17:45:22.166Z
[junit4] 2> 3016892 ERROR (jetty-launcher-3541-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3016893 INFO (jetty-launcher-3541-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 3016893 INFO (jetty-launcher-3541-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3016894 INFO (jetty-launcher-3541-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3016894 INFO (jetty-launcher-3541-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-04T17:45:22.168Z
[junit4] 2> 3016894 INFO (jetty-launcher-3541-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3016895 INFO (jetty-launcher-3541-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3016896 INFO (jetty-launcher-3541-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3016897 INFO (jetty-launcher-3541-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3016912 INFO (jetty-launcher-3541-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:63040/solr
[junit4] 2> 3016913 INFO (jetty-launcher-3541-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:63040/solr
[junit4] 2> 3016914 INFO (jetty-launcher-3541-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:63040/solr
[junit4] 2> 3016919 INFO (jetty-launcher-3541-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:63040/solr
[junit4] 2> 3016976 INFO (jetty-launcher-3541-thread-1) [n:127.0.0.1:63046_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:63046_solr
[junit4] 2> 3016978 INFO (jetty-launcher-3541-thread-2) [n:127.0.0.1:63042_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63042_solr
[junit4] 2> 3016978 INFO (jetty-launcher-3541-thread-4) [n:127.0.0.1:63043_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63043_solr
[junit4] 2> 3016978 INFO (jetty-launcher-3541-thread-3) [n:127.0.0.1:63044_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63044_solr
[junit4] 2> 3016979 INFO (jetty-launcher-3541-thread-1) [n:127.0.0.1:63046_solr ] o.a.s.c.Overseer Overseer (id=97401590385147916-127.0.0.1:63046_solr-n_0000000000) starting
[junit4] 2> 3016983 INFO (zkCallback-3560-thread-1-processing-n:127.0.0.1:63044_solr) [n:127.0.0.1:63044_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 3016983 INFO (zkCallback-3558-thread-1-processing-n:127.0.0.1:63043_solr) [n:127.0.0.1:63043_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 3016984 INFO (zkCallback-3561-thread-1-processing-n:127.0.0.1:63042_solr) [n:127.0.0.1:63042_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 3016984 INFO (zkCallback-3562-thread-1-processing-n:127.0.0.1:63046_solr) [n:127.0.0.1:63046_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 3017087 INFO (jetty-launcher-3541-thread-1) [n:127.0.0.1:63046_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63046_solr
[junit4] 2> 3017090 INFO (zkCallback-3562-thread-1-processing-n:127.0.0.1:63046_solr) [n:127.0.0.1:63046_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3017091 INFO (zkCallback-3560-thread-1-processing-n:127.0.0.1:63044_solr) [n:127.0.0.1:63044_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3017091 INFO (zkCallback-3561-thread-1-processing-n:127.0.0.1:63042_solr) [n:127.0.0.1:63042_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3017092 INFO (zkCallback-3558-thread-1-processing-n:127.0.0.1:63043_solr) [n:127.0.0.1:63043_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3017130 INFO (jetty-launcher-3541-thread-4) [n:127.0.0.1:63043_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/tempDir-001/node4/.
[junit4] 2> 3017132 INFO (jetty-launcher-3541-thread-3) [n:127.0.0.1:63044_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/tempDir-001/node3/.
[junit4] 2> 3017136 INFO (jetty-launcher-3541-thread-2) [n:127.0.0.1:63042_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/tempDir-001/node2/.
[junit4] 2> 3017249 INFO (jetty-launcher-3541-thread-1) [n:127.0.0.1:63046_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/tempDir-001/node1/.
[junit4] 2> 3020214 INFO (SUITE-CustomCollectionTest-seed#[F2B8AEDAE2C0C2AE]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 3020216 INFO (SUITE-CustomCollectionTest-seed#[F2B8AEDAE2C0C2AE]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:63040/solr ready
[junit4] 2> 3020256 INFO (TEST-CustomCollectionTest.testCreateShardRepFactor-seed#[F2B8AEDAE2C0C2AE]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCreateShardRepFactor
[junit4] 2> 3020259 INFO (TEST-CustomCollectionTest.testCreateShardRepFactor-seed#[F2B8AEDAE2C0C2AE]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 3020264 INFO (qtp366485284-17982) [n:127.0.0.1:63046_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params shards=a,b&replicationFactor=1&collection.configName=conf&name=testCreateShardRepFactor&router.name=implicit&action=CREATE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3020268 INFO (OverseerThreadFactory-9293-thread-1-processing-n:127.0.0.1:63046_solr) [n:127.0.0.1:63046_solr ] o.a.s.c.CreateCollectionCmd Create collection testCreateShardRepFactor
[junit4] 2> 3020382 INFO (qtp1492104561-17969) [n:127.0.0.1:63042_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=testCreateShardRepFactor_a_replica1&action=CREATE&numShards=2&collection=testCreateShardRepFactor&shard=a&wt=javabin&version=2
[junit4] 2> 3020382 INFO (qtp766890141-17987) [n:127.0.0.1:63044_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=testCreateShardRepFactor_b_replica1&action=CREATE&numShards=2&collection=testCreateShardRepFactor&shard=b&wt=javabin&version=2
[junit4] 2> 3020500 INFO (zkCallback-3561-thread-1-processing-n:127.0.0.1:63042_solr) [n:127.0.0.1:63042_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3020501 INFO (zkCallback-3560-thread-1-processing-n:127.0.0.1:63044_solr) [n:127.0.0.1:63044_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3021407 INFO (qtp766890141-17987) [n:127.0.0.1:63044_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 3021407 INFO (qtp1492104561-17969) [n:127.0.0.1:63042_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 3021415 INFO (qtp1492104561-17969) [n:127.0.0.1:63042_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.s.IndexSchema [testCreateShardRepFactor_a_replica1] Schema name=cloud-dynamic
[junit4] 2> 3021415 INFO (qtp766890141-17987) [n:127.0.0.1:63044_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.s.IndexSchema [testCreateShardRepFactor_b_replica1] Schema name=cloud-dynamic
[junit4] 2> 3021451 WARN (qtp766890141-17987) [n:127.0.0.1:63044_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.s.IndexSchema [testCreateShardRepFactor_b_replica1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 3021451 WARN (qtp1492104561-17969) [n:127.0.0.1:63042_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.s.IndexSchema [testCreateShardRepFactor_a_replica1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 3021452 INFO (qtp766890141-17987) [n:127.0.0.1:63044_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 3021452 INFO (qtp1492104561-17969) [n:127.0.0.1:63042_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 3021455 INFO (qtp766890141-17987) [n:127.0.0.1:63044_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.c.CoreContainer Creating SolrCore 'testCreateShardRepFactor_b_replica1' using configuration from collection testCreateShardRepFactor
[junit4] 2> 3021455 INFO (qtp1492104561-17969) [n:127.0.0.1:63042_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.c.CoreContainer Creating SolrCore 'testCreateShardRepFactor_a_replica1' using configuration from collection testCreateShardRepFactor
[junit4] 2> 3021455 INFO (qtp766890141-17987) [n:127.0.0.1:63044_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.c.SolrCore [[testCreateShardRepFactor_b_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/tempDir-001/node3/testCreateShardRepFactor_b_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/tempDir-001/node3/./testCreateShardRepFactor_b_replica1/data/]
[junit4] 2> 3021455 INFO (qtp1492104561-17969) [n:127.0.0.1:63042_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.c.SolrCore [[testCreateShardRepFactor_a_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/tempDir-001/node2/testCreateShardRepFactor_a_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/tempDir-001/node2/./testCreateShardRepFactor_a_replica1/data/]
[junit4] 2> 3021535 INFO (qtp1492104561-17969) [n:127.0.0.1:63042_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3021535 INFO (qtp1492104561-17969) [n:127.0.0.1:63042_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3021535 INFO (qtp766890141-17987) [n:127.0.0.1:63044_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3021535 INFO (qtp766890141-17987) [n:127.0.0.1:63044_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3021537 INFO (qtp1492104561-17969) [n:127.0.0.1:63042_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3021537 INFO (qtp1492104561-17969) [n:127.0.0.1:63042_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3021537 INFO (qtp766890141-17987) [n:127.0.0.1:63044_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3021537 INFO (qtp766890141-17987) [n:127.0.0.1:63044_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3021538 INFO (qtp1492104561-17969) [n:127.0.0.1:63042_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@12d04b01[testCreateShardRepFactor_a_replica1] main]
[junit4] 2> 3021539 INFO (qtp766890141-17987) [n:127.0.0.1:63044_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4db1053d[testCreateShardRepFactor_b_replica1] main]
[junit4] 2> 3021540 INFO (qtp1492104561-17969) [n:127.0.0.1:63042_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 3021541 INFO (qtp766890141-17987) [n:127.0.0.1:63044_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 3021541 INFO (qtp1492104561-17969) [n:127.0.0.1:63042_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 3021541 INFO (qtp1492104561-17969) [n:127.0.0.1:63042_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 3021541 INFO (qtp1492104561-17969) [n:127.0.0.1:63042_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558425451170365440
[junit4] 2> 3021541 INFO (qtp766890141-17987) [n:127.0.0.1:63044_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 3021541 INFO (searcherExecutor-9302-thread-1-processing-n:127.0.0.1:63042_solr x:testCreateShardRepFactor_a_replica1 s:a c:testCreateShardRepFactor) [n:127.0.0.1:63042_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.c.SolrCore [testCreateShardRepFactor_a_replica1] Registered new searcher Searcher@12d04b01[testCreateShardRepFactor_a_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3021541 INFO (qtp766890141-17987) [n:127.0.0.1:63044_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 3021542 INFO (qtp766890141-17987) [n:127.0.0.1:63044_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558425451171414016
[junit4] 2> 3021542 INFO (searcherExecutor-9301-thread-1-processing-n:127.0.0.1:63044_solr x:testCreateShardRepFactor_b_replica1 s:b c:testCreateShardRepFactor) [n:127.0.0.1:63044_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.c.SolrCore [testCreateShardRepFactor_b_replica1] Registered new searcher Searcher@4db1053d[testCreateShardRepFactor_b_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3021551 INFO (qtp1492104561-17969) [n:127.0.0.1:63042_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 3021551 INFO (qtp1492104561-17969) [n:127.0.0.1:63042_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 3021552 INFO (qtp1492104561-17969) [n:127.0.0.1:63042_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:63042/solr/testCreateShardRepFactor_a_replica1/
[junit4] 2> 3021552 INFO (qtp766890141-17987) [n:127.0.0.1:63044_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 3021552 INFO (qtp1492104561-17969) [n:127.0.0.1:63042_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 3021552 INFO (qtp766890141-17987) [n:127.0.0.1:63044_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 3021552 INFO (qtp766890141-17987) [n:127.0.0.1:63044_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:63044/solr/testCreateShardRepFactor_b_replica1/
[junit4] 2> 3021552 INFO (qtp1492104561-17969) [n:127.0.0.1:63042_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.c.SyncStrategy http://127.0.0.1:63042/solr/testCreateShardRepFactor_a_replica1/ has no replicas
[junit4] 2> 3021552 INFO (qtp766890141-17987) [n:127.0.0.1:63044_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 3021552 INFO (qtp766890141-17987) [n:127.0.0.1:63044_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.c.SyncStrategy http://127.0.0.1:63044/solr/testCreateShardRepFactor_b_replica1/ has no replicas
[junit4] 2> 3021560 INFO (qtp1492104561-17969) [n:127.0.0.1:63042_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:63042/solr/testCreateShardRepFactor_a_replica1/ a
[junit4] 2> 3021560 INFO (qtp766890141-17987) [n:127.0.0.1:63044_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:63044/solr/testCreateShardRepFactor_b_replica1/ b
[junit4] 2> 3021677 INFO (zkCallback-3561-thread-1-processing-n:127.0.0.1:63042_solr) [n:127.0.0.1:63042_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3021677 INFO (zkCallback-3560-thread-1-processing-n:127.0.0.1:63044_solr) [n:127.0.0.1:63044_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3021722 INFO (qtp766890141-17987) [n:127.0.0.1:63044_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 3021722 INFO (qtp1492104561-17969) [n:127.0.0.1:63042_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 3021726 INFO (qtp766890141-17987) [n:127.0.0.1:63044_solr c:testCreateShardRepFactor s:b x:testCreateShardRepFactor_b_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=testCreateShardRepFactor_b_replica1&action=CREATE&numShards=2&collection=testCreateShardRepFactor&shard=b&wt=javabin&version=2} status=0 QTime=1343
[junit4] 2> 3021726 INFO (qtp1492104561-17969) [n:127.0.0.1:63042_solr c:testCreateShardRepFactor s:a x:testCreateShardRepFactor_a_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=testCreateShardRepFactor_a_replica1&action=CREATE&numShards=2&collection=testCreateShardRepFactor&shard=a&wt=javabin&version=2} status=0 QTime=1343
[junit4] 2> 3021731 INFO (qtp366485284-17982) [n:127.0.0.1:63046_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 3021845 INFO (zkCallback-3560-thread-1-processing-n:127.0.0.1:63044_solr) [n:127.0.0.1:63044_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3021845 INFO (zkCallback-3561-thread-1-processing-n:127.0.0.1:63042_solr) [n:127.0.0.1:63042_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3022283 INFO (OverseerCollectionConfigSetProcessor-97401590385147916-127.0.0.1:63046_solr-n_0000000000) [n:127.0.0.1:63046_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 3022734 INFO (qtp366485284-17982) [n:127.0.0.1:63046_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={shards=a,b&replicationFactor=1&collection.configName=conf&name=testCreateShardRepFactor&router.name=implicit&action=CREATE&wt=javabin&version=2} status=0 QTime=2470
[junit4] 2> 3022735 INFO (qtp366485284-17983) [n:127.0.0.1:63046_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createshard with params action=CREATESHARD&collection=testCreateShardRepFactor&shard=x&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3022740 INFO (OverseerThreadFactory-9293-thread-2-processing-n:127.0.0.1:63046_solr) [n:127.0.0.1:63046_solr ] o.a.s.c.CreateShardCmd Create shard invoked: {
[junit4] 2> "collection":"testCreateShardRepFactor",
[junit4] 2> "shard":"x",
[junit4] 2> "operation":"createshard"}
[junit4] 2> 3022860 INFO (zkCallback-3561-thread-1-processing-n:127.0.0.1:63042_solr) [n:127.0.0.1:63042_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3022860 INFO (zkCallback-3560-thread-1-processing-n:127.0.0.1:63044_solr) [n:127.0.0.1:63044_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3022949 INFO (OverseerThreadFactory-9293-thread-2-processing-n:127.0.0.1:63046_solr) [n:127.0.0.1:63046_solr ] o.a.s.c.CreateShardCmd Creating shard testCreateShardRepFactor_x_replica1 as part of slice x of collection testCreateShardRepFactor on 127.0.0.1:63046_solr
[junit4] 2> 3022951 INFO (qtp366485284-17989) [n:127.0.0.1:63046_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&name=testCreateShardRepFactor_x_replica1&action=CREATE&numShards=1&collection=testCreateShardRepFactor&shard=x&wt=javabin&version=2
[junit4] 2> 3023061 INFO (zkCallback-3560-thread-1-processing-n:127.0.0.1:63044_solr) [n:127.0.0.1:63044_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3023061 INFO (zkCallback-3562-thread-1-processing-n:127.0.0.1:63046_solr) [n:127.0.0.1:63046_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3023061 INFO (zkCallback-3561-thread-1-processing-n:127.0.0.1:63042_solr) [n:127.0.0.1:63042_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3023978 INFO (qtp366485284-17989) [n:127.0.0.1:63046_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 3024004 INFO (qtp366485284-17989) [n:127.0.0.1:63046_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.s.IndexSchema [testCreateShardRepFactor_x_replica1] Schema name=cloud-dynamic
[junit4] 2> 3024024 WARN (qtp366485284-17989) [n:127.0.0.1:63046_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.s.IndexSchema [testCreateShardRepFactor_x_replica1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 3024025 INFO (qtp366485284-17989) [n:127.0.0.1:63046_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 3024026 INFO (qtp366485284-17989) [n:127.0.0.1:63046_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.c.CoreContainer Creating SolrCore 'testCreateShardRepFactor_x_replica1' using configuration from collection testCreateShardRepFactor
[junit4] 2> 3024027 INFO (qtp366485284-17989) [n:127.0.0.1:63046_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.c.SolrCore [[testCreateShardRepFactor_x_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/tempDir-001/node1/testCreateShardRepFactor_x_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/tempDir-001/node1/./testCreateShardRepFactor_x_replica1/data/]
[junit4] 2> 3024095 INFO (qtp366485284-17989) [n:127.0.0.1:63046_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3024095 INFO (qtp366485284-17989) [n:127.0.0.1:63046_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3024097 INFO (qtp366485284-17989) [n:127.0.0.1:63046_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3024097 INFO (qtp366485284-17989) [n:127.0.0.1:63046_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3024098 INFO (qtp366485284-17989) [n:127.0.0.1:63046_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@32528d41[testCreateShardRepFactor_x_replica1] main]
[junit4] 2> 3024099 INFO (qtp366485284-17989) [n:127.0.0.1:63046_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 3024100 INFO (qtp366485284-17989) [n:127.0.0.1:63046_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 3024100 INFO (qtp366485284-17989) [n:127.0.0.1:63046_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 3024100 INFO (qtp366485284-17989) [n:127.0.0.1:63046_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558425453853671424
[junit4] 2> 3024101 INFO (searcherExecutor-9311-thread-1-processing-n:127.0.0.1:63046_solr x:testCreateShardRepFactor_x_replica1 s:x c:testCreateShardRepFactor) [n:127.0.0.1:63046_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.c.SolrCore [testCreateShardRepFactor_x_replica1] Registered new searcher Searcher@32528d41[testCreateShardRepFactor_x_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3024110 INFO (qtp366485284-17989) [n:127.0.0.1:63046_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 3024111 INFO (qtp366485284-17989) [n:127.0.0.1:63046_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 3024111 INFO (qtp366485284-17989) [n:127.0.0.1:63046_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:63046/solr/testCreateShardRepFactor_x_replica1/
[junit4] 2> 3024111 INFO (qtp366485284-17989) [n:127.0.0.1:63046_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 3024111 INFO (qtp366485284-17989) [n:127.0.0.1:63046_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.c.SyncStrategy http://127.0.0.1:63046/solr/testCreateShardRepFactor_x_replica1/ has no replicas
[junit4] 2> 3024118 INFO (qtp366485284-17989) [n:127.0.0.1:63046_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:63046/solr/testCreateShardRepFactor_x_replica1/ x
[junit4] 2> 3024224 INFO (zkCallback-3561-thread-1-processing-n:127.0.0.1:63042_solr) [n:127.0.0.1:63042_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3024224 INFO (zkCallback-3560-thread-1-processing-n:127.0.0.1:63044_solr) [n:127.0.0.1:63044_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3024224 INFO (zkCallback-3562-thread-3-processing-n:127.0.0.1:63046_solr) [n:127.0.0.1:63046_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCreateShardRepFactor/state.json] for collection [testCreateShardRepFactor] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3024229 INFO (qtp366485284-17989) [n:127.0.0.1:63046_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 3024232 INFO (qtp366485284-17989) [n:127.0.0.1:63046_solr c:testCreateShardRepFactor s:x x:testCreateShardRepFactor_x_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&name=testCreateShardRepFactor_x_replica1&action=CREATE&numShards=1&collection=testCreateShardRepFactor&shard=x&wt=javabin&version=2} status=0 QTime=1281
[junit4] 2> 3024232 INFO (OverseerThreadFactory-9293-thread-2-processing-n:127.0.0.1:63046_solr) [n:127.0.0.1:63046_solr ] o.a.s.c.CreateShardCmd Finished create command on all shards for collection: testCreateShardRepFactor
[junit4] 2> 3024245 INFO (qtp366485284-17983) [n:127.0.0.1:63046_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=CREATESHARD&collection=testCreateShardRepFactor&shard=x&wt=javabin&version=2} status=0 QTime=1509
[junit4] 2> 3024247 INFO (TEST-CustomCollectionTest.testCreateShardRepFactor-seed#[F2B8AEDAE2C0C2AE]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testCreateShardRepFactor
[junit4] 2> 3024279 INFO (TEST-CustomCollectionTest.testRouteFieldForHashRouter-seed#[F2B8AEDAE2C0C2AE]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testRouteFieldForHashRouter
[junit4] 2> 3024282 INFO (TEST-CustomCollectionTest.testRouteFieldForHashRouter-seed#[F2B8AEDAE2C0C2AE]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 3024284 INFO (qtp366485284-17990) [n:127.0.0.1:63046_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=testCreateShardRepFactor&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3024289 INFO (OverseerThreadFactory-9293-thread-3-processing-n:127.0.0.1:63046_solr) [n:127.0.0.1:63046_solr ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
[junit4] 2> 3024289 INFO (OverseerCollectionConfigSetProcessor-97401590385147916-127.0.0.1:63046_solr-n_0000000000) [n:127.0.0.1:63046_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 3024290 INFO (qtp1492104561-17970) [n:127.0.0.1:63042_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.testCreateShardRepFactor.a.replica1
[junit4] 2> 3024290 INFO (qtp766890141-17988) [n:127.0.0.1:63044_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.testCreateShardRepFactor.b.replica1
[junit4] 2> 3024290 INFO (qtp366485284-17991) [n:127.0.0.1:63046_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.testCreateShardRepFactor.x.replica1
[junit4] 2> 3024290 INFO (qtp766890141-17988) [n:127.0.0.1:63044_solr ] o.a.s.c.SolrCore [testCreateShardRepFactor_b_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@72a6cc81
[junit4] 2> 3024291 INFO (qtp366485284-17991) [n:127.0.0.1:63046_solr ] o.a.s.c.SolrCore [testCreateShardRepFactor_x_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@4b24625f
[junit4] 2> 3024290 INFO (qtp1492104561-17970) [n:127.0.0.1:63042_solr ] o.a.s.c.SolrCore [testCreateShardRepFactor_a_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@19559d46
[junit4] 2> 3024291 INFO (qtp366485284-17991) [n:127.0.0.1:63046_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.testCreateShardRepFactor.x.replica1
[junit4] 2> 3024291 INFO (qtp1492104561-17970) [n:127.0.0.1:63042_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.testCreateShardRepFactor.a.replica1
[junit4] 2> 3024291 INFO (qtp766890141-17988) [n:127.0.0.1:63044_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.testCreateShardRepFactor.b.replica1
[junit4] 2> 3024296 INFO (qtp366485284-17991) [n:127.0.0.1:63046_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=testCreateShardRepFactor_x_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=5
[junit4] 2> 3024296 INFO (qtp1492104561-17970) [n:127.0.0.1:63042_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=testCreateShardRepFactor_a_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=6
[junit4] 2> 3024297 INFO (qtp766890141-17988) [n:127.0.0.1:63044_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=testCreateShardRepFactor_b_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=6
[junit4] 2> 3025034 INFO (qtp366485284-17990) [n:127.0.0.1:63046_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=testCreateShardRepFactor&action=DELETE&wt=javabin&version=2} status=0 QTime=749
[junit4] 2> 3025035 INFO (qtp366485284-17965) [n:127.0.0.1:63046_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&maxShardsPerNode=3&router.field=shard_s&name=routeFieldColl&action=CREATE&numShards=4&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3025039 INFO (OverseerThreadFactory-9293-thread-4-processing-n:127.0.0.1:63046_solr) [n:127.0.0.1:63046_solr ] o.a.s.c.CreateCollectionCmd Create collection routeFieldColl
[junit4] 2> 3025040 INFO (OverseerCollectionConfigSetProcessor-97401590385147916-127.0.0.1:63046_solr-n_0000000000) [n:127.0.0.1:63046_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 3025261 INFO (qtp366485284-17982) [n:127.0.0.1:63046_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=routeFieldColl_shard4_replica1&action=CREATE&numShards=4&collection=routeFieldColl&shard=shard4&wt=javabin&version=2
[junit4] 2> 3025262 INFO (qtp366485284-17985) [n:127.0.0.1:63046_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=routeFieldColl_shard2_replica1&action=CREATE&numShards=4&collection=routeFieldColl&shard=shard2&wt=javabin&version=2
[junit4] 2> 3025265 INFO (qtp766890141-17994) [n:127.0.0.1:63044_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=routeFieldColl_shard1_replica1&action=CREATE&numShards=4&collection=routeFieldColl&shard=shard1&wt=javabin&version=2
[junit4] 2> 3025268 INFO (qtp1492104561-17971) [n:127.0.0.1:63042_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=routeFieldColl_shard3_replica2&action=CREATE&numShards=4&collection=routeFieldColl&shard=shard3&wt=javabin&version=2
[junit4] 2> 3025271 INFO (qtp766890141-17996) [n:127.0.0.1:63044_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=routeFieldColl_shard3_replica1&action=CREATE&numShards=4&collection=routeFieldColl&shard=shard3&wt=javabin&version=2
[junit4] 2> 3025273 INFO (qtp233442276-17978) [n:127.0.0.1:63043_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=routeFieldColl_shard2_replica2&action=CREATE&numShards=4&collection=routeFieldColl&shard=shard2&wt=javabin&version=2
[junit4] 2> 3025274 INFO (qtp1492104561-17972) [n:127.0.0.1:63042_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=routeFieldColl_shard1_replica2&action=CREATE&numShards=4&collection=routeFieldColl&shard=shard1&wt=javabin&version=2
[junit4] 2> 3025273 INFO (qtp233442276-17981) [n:127.0.0.1:63043_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=routeFieldColl_shard4_replica2&action=CREATE&numShards=4&collection=routeFieldColl&shard=shard4&wt=javabin&version=2
[junit4] 2> 3025420 INFO (zkCallback-3560-thread-1-processing-n:127.0.0.1:63044_solr) [n:127.0.0.1:63044_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routeFieldColl/state.json] for collection [routeFieldColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3025420 INFO (zkCallback-3558-thread-1-processing-n:127.0.0.1:63043_solr) [n:127.0.0.1:63043_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routeFieldColl/state.json] for collection [routeFieldColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3025420 INFO (zkCallback-3562-thread-3-processing-n:127.0.0.1:63046_solr) [n:127.0.0.1:63046_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routeFieldColl/state.json] for collection [routeFieldColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3025420 INFO (zkCallback-3561-thread-1-processing-n:127.0.0.1:63042_solr) [n:127.0.0.1:63042_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/routeFieldColl/state.json] for collection [routeFieldColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3026306 INFO (qtp366485284-17985) [n:127.0.0.1:63046_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 3026332 INFO (qtp366485284-17982) [n:127.0.0.1:63046_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 3026335 INFO (qtp1492104561-17971) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 3026335 INFO (qtp233442276-17981) [n:127.0.0.1:63043_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 3026340 INFO (qtp1492104561-17972) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 3026363 INFO (qtp233442276-17978) [n:127.0.0.1:63043_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 3026366 INFO (qtp766890141-17994) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 3026368 INFO (qtp766890141-17996) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 3026374 INFO (qtp1492104561-17971) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.s.IndexSchema [routeFieldColl_shard3_replica2] Schema name=cloud-dynamic
[junit4] 2> 3026376 INFO (qtp766890141-17994) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.s.IndexSchema [routeFieldColl_shard1_replica1] Schema name=cloud-dynamic
[junit4] 2> 3026376 INFO (qtp233442276-17981) [n:127.0.0.1:63043_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica2] o.a.s.s.IndexSchema [routeFieldColl_shard4_replica2] Schema name=cloud-dynamic
[junit4] 2> 3026390 INFO (qtp1492104561-17972) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.s.IndexSchema [routeFieldColl_shard1_replica2] Schema name=cloud-dynamic
[junit4] 2> 3026401 INFO (qtp233442276-17978) [n:127.0.0.1:63043_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.s.IndexSchema [routeFieldColl_shard2_replica2] Schema name=cloud-dynamic
[junit4] 2> 3026401 INFO (qtp766890141-17996) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.s.IndexSchema [routeFieldColl_shard3_replica1] Schema name=cloud-dynamic
[junit4] 2> 3026431 INFO (qtp366485284-17982) [n:127.0.0.1:63046_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.s.IndexSchema [routeFieldColl_shard4_replica1] Schema name=cloud-dynamic
[junit4] 2> 3026448 INFO (qtp366485284-17985) [n:127.0.0.1:63046_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.s.IndexSchema [routeFieldColl_shard2_replica1] Schema name=cloud-dynamic
[junit4] 2> 3026448 WARN (qtp1492104561-17971) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.s.IndexSchema [routeFieldColl_shard3_replica2] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 3026449 INFO (qtp1492104561-17971) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 3026451 INFO (qtp1492104561-17971) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.c.CoreContainer Creating SolrCore 'routeFieldColl_shard3_replica2' using configuration from collection routeFieldColl
[junit4] 2> 3026451 INFO (qtp1492104561-17971) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.c.SolrCore [[routeFieldColl_shard3_replica2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/tempDir-001/node2/routeFieldColl_shard3_replica2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/tempDir-001/node2/./routeFieldColl_shard3_replica2/data/]
[junit4] 2> 3026453 WARN (qtp1492104561-17972) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.s.IndexSchema [routeFieldColl_shard1_replica2] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 3026453 INFO (qtp1492104561-17972) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 3026454 INFO (qtp1492104561-17972) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.c.CoreContainer Creating SolrCore 'routeFieldColl_shard1_replica2' using configuration from collection routeFieldColl
[junit4] 2> 3026455 INFO (qtp1492104561-17972) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.c.SolrCore [[routeFieldColl_shard1_replica2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/tempDir-001/node2/routeFieldColl_shard1_replica2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/tempDir-001/node2/./routeFieldColl_shard1_replica2/data/]
[junit4] 2> 3026462 WARN (qtp766890141-17996) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.s.IndexSchema [routeFieldColl_shard3_replica1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 3026462 WARN (qtp233442276-17981) [n:127.0.0.1:63043_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica2] o.a.s.s.IndexSchema [routeFieldColl_shard4_replica2] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 3026463 INFO (qtp766890141-17996) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 3026463 INFO (qtp233442276-17981) [n:127.0.0.1:63043_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica2] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 3026465 INFO (qtp233442276-17981) [n:127.0.0.1:63043_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica2] o.a.s.c.CoreContainer Creating SolrCore 'routeFieldColl_shard4_replica2' using configuration from collection routeFieldColl
[junit4] 2> 3026465 INFO (qtp233442276-17981) [n:127.0.0.1:63043_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica2] o.a.s.c.SolrCore [[routeFieldColl_shard4_replica2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/tempDir-001/node4/routeFieldColl_shard4_replica2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/tempDir-001/node4/./routeFieldColl_shard4_replica2/data/]
[junit4] 2> 3026491 INFO (qtp766890141-17996) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.c.CoreContainer Creating SolrCore 'routeFieldColl_shard3_replica1' using configuration from collection routeFieldColl
[junit4] 2> 3026492 INFO (qtp766890141-17996) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.c.SolrCore [[routeFieldColl_shard3_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/tempDir-001/node3/routeFieldColl_shard3_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/tempDir-001/node3/./routeFieldColl_shard3_replica1/data/]
[junit4] 2> 3026493 WARN (qtp366485284-17982) [n:127.0.0.1:63046_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.s.IndexSchema [routeFieldColl_shard4_replica1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 3026493 INFO (qtp366485284-17982) [n:127.0.0.1:63046_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 3026495 INFO (qtp366485284-17982) [n:127.0.0.1:63046_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.c.CoreContainer Creating SolrCore 'routeFieldColl_shard4_replica1' using configuration from collection routeFieldColl
[junit4] 2> 3026495 INFO (qtp366485284-17982) [n:127.0.0.1:63046_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.c.SolrCore [[routeFieldColl_shard4_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/tempDir-001/node1/routeFieldColl_shard4_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/tempDir-001/node1/./routeFieldColl_shard4_replica1/data/]
[junit4] 2> 3026496 WARN (qtp233442276-17978) [n:127.0.0.1:63043_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.s.IndexSchema [routeFieldColl_shard2_replica2] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 3026497 INFO (qtp233442276-17978) [n:127.0.0.1:63043_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 3026498 INFO (qtp233442276-17978) [n:127.0.0.1:63043_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.c.CoreContainer Creating SolrCore 'routeFieldColl_shard2_replica2' using configuration from collection routeFieldColl
[junit4] 2> 3026498 INFO (qtp233442276-17978) [n:127.0.0.1:63043_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.c.SolrCore [[routeFieldColl_shard2_replica2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/tempDir-001/node4/routeFieldColl_shard2_replica2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/tempDir-001/node4/./routeFieldColl_shard2_replica2/data/]
[junit4] 2> 3026499 WARN (qtp766890141-17994) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.s.IndexSchema [routeFieldColl_shard1_replica1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 3026500 INFO (qtp766890141-17994) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 3026501 INFO (qtp766890141-17994) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'routeFieldColl_shard1_replica1' using configuration from collection routeFieldColl
[junit4] 2> 3026501 INFO (qtp766890141-17994) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.c.SolrCore [[routeFieldColl_shard1_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/tempDir-001/node3/routeFieldColl_shard1_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/tempDir-001/node3/./routeFieldColl_shard1_replica1/data/]
[junit4] 2> 3026502 WARN (qtp366485284-17985) [n:127.0.0.1:63046_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.s.IndexSchema [routeFieldColl_shard2_replica1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 3026502 INFO (qtp366485284-17985) [n:127.0.0.1:63046_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 3026504 INFO (qtp366485284-17985) [n:127.0.0.1:63046_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.c.CoreContainer Creating SolrCore 'routeFieldColl_shard2_replica1' using configuration from collection routeFieldColl
[junit4] 2> 3026505 INFO (qtp366485284-17985) [n:127.0.0.1:63046_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica1] o.a.s.c.SolrCore [[routeFieldColl_shard2_replica1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/tempDir-001/node1/routeFieldColl_shard2_replica1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CustomCollectionTest_F2B8AEDAE2C0C2AE-001/tempDir-001/node1/./routeFieldColl_shard2_replica1/data/]
[junit4] 2> 3026636 INFO (qtp366485284-17982) [n:127.0.0.1:63046_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3026637 INFO (qtp366485284-17982) [n:127.0.0.1:63046_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3026638 INFO (qtp366485284-17982) [n:127.0.0.1:63046_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3026639 INFO (qtp366485284-17982) [n:127.0.0.1:63046_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3026642 INFO (qtp366485284-17982) [n:127.0.0.1:63046_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@b155fdf[routeFieldColl_shard4_replica1] main]
[junit4] 2> 3026649 INFO (qtp766890141-17994) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3026649 INFO (qtp766890141-17994) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3026651 INFO (qtp766890141-17994) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3026651 INFO (qtp766890141-17994) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3026655 INFO (qtp766890141-17994) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7274453f[routeFieldColl_shard1_replica1] main]
[junit4] 2> 3026657 INFO (qtp766890141-17994) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 3026657 INFO (qtp1492104561-17972) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3026657 INFO (qtp1492104561-17972) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3026657 INFO (qtp766890141-17994) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 3026657 INFO (qtp766890141-17994) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 3026658 INFO (qtp766890141-17994) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558425456535928832
[junit4] 2> 3026658 INFO (searcherExecutor-9322-thread-1-processing-n:127.0.0.1:63044_solr x:routeFieldColl_shard1_replica1 s:shard1 c:routeFieldColl) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.c.SolrCore [routeFieldColl_shard1_replica1] Registered new searcher Searcher@7274453f[routeFieldColl_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3026659 INFO (qtp1492104561-17972) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3026659 INFO (qtp1492104561-17972) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3026662 INFO (qtp766890141-17996) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3026662 INFO (qtp766890141-17996) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3026663 INFO (qtp366485284-17982) [n:127.0.0.1:63046_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 3026664 INFO (qtp366485284-17982) [n:127.0.0.1:63046_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 3026664 INFO (qtp366485284-17982) [n:127.0.0.1:63046_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 3026664 INFO (searcherExecutor-9320-thread-1-processing-n:127.0.0.1:63046_solr x:routeFieldColl_shard4_replica1 s:shard4 c:routeFieldColl) [n:127.0.0.1:63046_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.c.SolrCore [routeFieldColl_shard4_replica1] Registered new searcher Searcher@b155fdf[routeFieldColl_shard4_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3026664 INFO (qtp366485284-17982) [n:127.0.0.1:63046_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558425456542220288
[junit4] 2> 3026670 INFO (qtp1492104561-17972) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@50ce9f13[routeFieldColl_shard1_replica2] main]
[junit4] 2> 3026674 INFO (qtp766890141-17994) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
[junit4] 2> 3026677 INFO (qtp1492104561-17971) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3026677 INFO (qtp1492104561-17971) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3026677 INFO (qtp766890141-17996) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3026677 INFO (qtp766890141-17996) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3026677 INFO (qtp233442276-17978) [n:127.0.0.1:63043_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3026677 INFO (qtp233442276-17978) [n:127.0.0.1:63043_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3026679 INFO (qtp1492104561-17971) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3026679 INFO (qtp1492104561-17971) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3026679 INFO (qtp233442276-17978) [n:127.0.0.1:63043_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3026679 INFO (qtp233442276-17978) [n:127.0.0.1:63043_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3026681 INFO (qtp233442276-17978) [n:127.0.0.1:63043_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@6181e1cf[routeFieldColl_shard2_replica2] main]
[junit4] 2> 3026682 INFO (qtp1492104561-17971) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@58609d3e[routeFieldColl_shard3_replica2] main]
[junit4] 2> 3026683 INFO (qtp233442276-17978) [n:127.0.0.1:63043_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 3026683 INFO (qtp766890141-17996) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@fe20745[routeFieldColl_shard3_replica1] main]
[junit4] 2> 3026684 INFO (qtp233442276-17978) [n:127.0.0.1:63043_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 3026684 INFO (qtp233442276-17978) [n:127.0.0.1:63043_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 3026684 INFO (qtp1492104561-17971) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 3026684 INFO (searcherExecutor-9321-thread-1-processing-n:127.0.0.1:63043_solr x:routeFieldColl_shard2_replica2 s:shard2 c:routeFieldColl) [n:127.0.0.1:63043_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.c.SolrCore [routeFieldColl_shard2_replica2] Registered new searcher Searcher@6181e1cf[routeFieldColl_shard2_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3026685 INFO (qtp233442276-17978) [n:127.0.0.1:63043_solr c:routeFieldColl s:shard2 x:routeFieldColl_shard2_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558425456564240384
[junit4] 2> 3026685 INFO (qtp233442276-17981) [n:127.0.0.1:63043_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3026685 INFO (qtp233442276-17981) [n:127.0.0.1:63043_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3026685 INFO (qtp1492104561-17971) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 3026685 INFO (qtp1492104561-17971) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 3026686 INFO (searcherExecutor-9316-thread-1-processing-n:127.0.0.1:63042_solr x:routeFieldColl_shard3_replica2 s:shard3 c:routeFieldColl) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.c.SolrCore [routeFieldColl_shard3_replica2] Registered new searcher Searcher@58609d3e[routeFieldColl_shard3_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3026686 INFO (qtp1492104561-17971) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558425456565288960
[junit4] 2> 3026687 INFO (qtp233442276-17981) [n:127.0.0.1:63043_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3026687 INFO (qtp233442276-17981) [n:127.0.0.1:63043_solr c:routeFieldColl s:shard4 x:routeFieldColl_shard4_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3026689 INFO (qtp1492104561-17972) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 3026690 INFO (qtp766890141-17996) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 3026691 INFO (qtp1492104561-17972) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 3026691 INFO (qtp1492104561-17972) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 3026692 INFO (searcherExecutor-9317-thread-1-processing-n:127.0.0.1:63042_solr x:routeFieldColl_shard1_replica2 s:shard1 c:routeFieldColl) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.c.SolrCore [routeFieldColl_shard1_replica2] Registered new searcher Searcher@50ce9f13[routeFieldColl_shard1_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3026692 INFO (qtp1492104561-17972) [n:127.0.0.1:63042_solr c:routeFieldColl s:shard1 x:routeFieldColl_shard1_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558425456571580416
[junit4] 2> 3026693 INFO (qtp766890141-17996) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 3026693 INFO (qtp766890141-17996) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 3026693 INFO (searcherExecutor-9319-thread-1-processing-n:127.0.0.1:63044_solr x:routeFieldColl_shard3_replica1 s:shard3 c:routeFieldColl) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.c.SolrCore [routeFieldColl_shard3_replica1] Registered new searcher Searcher@fe20745[routeFieldColl_shard3_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3026693 INFO (qtp766890141-17996) [n:127.0.0.1:63044_solr c:routeFieldColl s:shard3 x:routeFieldColl_shard3_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558425456572628992
[junit4] 2> 3026694 INFO (qtp233442276-179
[...truncated too long message...]
2> 5 /solr/collections
[junit4] 2> 4 /solr/overseer/queue-work
[junit4] 2>
[junit4] 2> 3174014 WARN (TEST-HttpPartitionTest.test-seed#[F2B8AEDAE2C0C2AE]) [ ] o.a.s.c.SocketProxy Closing 6 connections to: http://127.0.0.1:63333/, target: http://127.0.0.1:63334/
[junit4] 2> 3174014 WARN (TEST-HttpPartitionTest.test-seed#[F2B8AEDAE2C0C2AE]) [ ] o.a.s.c.SocketProxy Closing 3 connections to: http://127.0.0.1:63308/, target: http://127.0.0.1:63309/
[junit4] 2> 3174015 WARN (TEST-HttpPartitionTest.test-seed#[F2B8AEDAE2C0C2AE]) [ ] o.a.s.c.SocketProxy Closing 4 connections to: http://127.0.0.1:63340/, target: http://127.0.0.1:63341/
[junit4] 2> 3174015 WARN (TEST-HttpPartitionTest.test-seed#[F2B8AEDAE2C0C2AE]) [ ] o.a.s.c.SocketProxy Closing 5 connections to: http://127.0.0.1:63322/, target: http://127.0.0.1:63323/
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=HttpPartitionTest -Dtests.method=test -Dtests.seed=F2B8AEDAE2C0C2AE -Dtests.slow=true -Dtests.locale=fr-LU -Dtests.timezone=Asia/Karachi -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 63.4s J1 | HttpPartitionTest.test <<<
[junit4] > Throwable #1: org.apache.solr.common.SolrException: Collection not found: collMinRf_1x3
[junit4] > at __randomizedtesting.SeedInfo.seed([F2B8AEDAE2C0C2AE:7AEC91004C3CAF56]:0)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.getCollectionNames(CloudSolrClient.java:1376)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1072)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1042)
[junit4] > at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
[junit4] > at org.apache.solr.cloud.HttpPartitionTest.sendDoc(HttpPartitionTest.java:611)
[junit4] > at org.apache.solr.cloud.HttpPartitionTest.testMinRf(HttpPartitionTest.java:210)
[junit4] > at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:120)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 3174016 INFO (SUITE-HttpPartitionTest-seed#[F2B8AEDAE2C0C2AE]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_F2B8AEDAE2C0C2AE-001
[junit4] 2> NOTE: test params are: codec=Lucene70, sim=RandomSimilarity(queryNorm=false): {}, locale=fr-LU, timezone=Asia/Karachi
[junit4] 2> NOTE: Mac OS X 10.11.6 x86_64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=3,threads=1,free=317376312,total=530055168
[junit4] 2> NOTE: All tests run in this JVM: [TestQueryUtils, TestIndexingPerformance, TestLMJelinekMercerSimilarityFactory, TestDocSet, ResponseLogComponentTest, HdfsSyncSliceTest, TestSolrCoreSnapshots, RemoteQueryErrorTest, BasicDistributedZk2Test, TestSolrConfigHandlerCloud, ZkSolrClientTest, LeaderFailoverAfterPartitionTest, TestFuzzyAnalyzedSuggestions, TestSolrFieldCacheMBean, FieldAnalysisRequestHandlerTest, TestDocTermOrds, SolrCmdDistributorTest, BlockJoinFacetSimpleTest, RegexBoostProcessorTest, CdcrUpdateLogTest, JsonLoaderTest, TestDynamicFieldResource, SolrCloudExampleTest, TestDeleteCollectionOnDownNodes, DistributedDebugComponentTest, SuggesterTSTTest, BlockJoinFacetDistribTest, SolrCoreMetricManagerTest, UpdateParamsTest, SolrJmxReporterTest, CloneFieldUpdateProcessorFactoryTest, TestExportWriter, TestCustomSort, ExplicitHLLTest, HdfsNNFailoverTest, TestRandomFaceting, AnalyticsQueryTest, TestHdfsCloudBackupRestore, HdfsRecoveryZkTest, TestPivotHelperCode, TestValueSourceCache, TestQueryTypes, DistributedFacetExistsSmallTest, TestLeaderElectionZkExpiry, CollectionsAPIAsyncDistributedZkTest, TestRawTransformer, TestUtils, DistributedQueueTest, ConnectionManagerTest, SegmentsInfoRequestHandlerTest, TestSystemIdResolver, TestDFRSimilarityFactory, TestStressInPlaceUpdates, TestSolrQueryParserDefaultOperatorResource, SolrIndexSplitterTest, FieldMutatingUpdateProcessorTest, TestPerFieldSimilarityWithDefaultOverride, TestReloadAndDeleteDocs, TestRandomFlRTGCloud, RollingRestartTest, TestObjectReleaseTracker, TestReversedWildcardFilterFactory, TestFoldingMultitermQuery, CircularListTest, LeaderInitiatedRecoveryOnCommitTest, JavabinLoaderTest, MetricsHandlerTest, TestSQLHandler, TestMiniSolrCloudCluster, TestCrossCoreJoin, FileBasedSpellCheckerTest, HdfsChaosMonkeySafeLeaderTest, TestFieldCacheVsDocValues, MigrateRouteKeyTest, DistributedTermsComponentTest, TestConfigSets, HdfsWriteToMultipleCollectionsTest, TestSchemalessBufferedUpdates, NoCacheHeaderTest, TestFieldResource, TestManagedResource, RequestHandlersTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TermVectorComponentDistributedTest, SimpleCollectionCreateDeleteTest, ReplicationFactorTest, DebugComponentTest, SliceStateTest, TestRecovery, CSVRequestHandlerTest, AtomicUpdatesTest, TestStressRecovery, TestFieldTypeResource, DistribDocExpirationUpdateProcessorTest, ZkStateWriterTest, MultiThreadedOCPTest, CdcrRequestHandlerTest, PreAnalyzedUpdateProcessorTest, DocValuesNotIndexedTest, TestLuceneMatchVersion, VersionInfoTest, TestTrie, TestReloadDeadlock, TestConfigsApi, DirectoryFactoryTest, TestLRUStatsCache, CoreMergeIndexesAdminHandlerTest, CurrencyFieldOpenExchangeTest, TestSimpleTrackingShardHandler, BasicZkTest, SpellPossibilityIteratorTest, TestJettySolrRunner, LoggingHandlerTest, TestSimpleQParserPlugin, EchoParamsTest, SparseHLLTest, BaseCdcrDistributedZkTest, UpdateRequestProcessorFactoryTest, TestAuthorizationFramework, TestFileDictionaryLookup, TestBinaryResponseWriter, TestIndexSearcher, TestNumericTerms64, FullSolrCloudDistribCmdsTest, TestHdfsBackupRestoreCore, TermVectorComponentTest, DeleteLastCustomShardedReplicaTest, TestSolrIndexConfig, TestFieldCacheReopen, SortSpecParsingTest, TestRangeQuery, BlobRepositoryCloudTest, DistributedQueryComponentCustomSortTest, TestDistributedGrouping, TestComponentsName, TestImplicitCoreProperties, SuggestComponentContextFilterQueryTest, TestRestManager, VMParamsZkACLAndCredentialsProvidersTest, TestDistributedMissingSort, BlockDirectoryTest, TestManagedSynonymFilterFactory, TestAuthenticationFramework, SolrInfoMBeanTest, SuggesterTest, TestXmlQParser, ShowFileRequestHandlerTest, TestPHPSerializedResponseWriter, HLLSerializationTest, DefaultValueUpdateProcessorTest, TestQuerySenderListener, TestJsonFacetRefinement, TestFiltering, ConvertedLegacyTest, TestReqParamsAPI, TestSolr4Spatial2, TestInitParams, TestCloudPseudoReturnFields, PreAnalyzedFieldTest, TestConfigSetsAPI, ReturnFieldsTest, SharedFSAutoReplicaFailoverTest, TestOverriddenPrefixQueryForCustomFieldType, HdfsDirectoryTest, PeerSyncWithIndexFingerprintCachingTest, DistributedFacetPivotLongTailTest, TestDFISimilarityFactory, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest, OpenCloseCoreStressTest, LeaderElectionIntegrationTest, RecoveryZkTest, ClusterStateUpdateTest, LeaderElectionTest, TestZkChroot, ZkCLITest, ShardRoutingCustomTest, TestDistributedSearch, AutoCommitTest, TestFaceting, TestHashPartitioner, ZkControllerTest, TestRealTimeGet, TestJoin, TestReload, TestCoreContainer, TestSolr4Spatial, SolrCoreTest, StatsComponentTest, SpellCheckComponentTest, TestGroupingSearch, QueryElevationComponentTest, BadIndexSchemaTest, TestFunctionQuery, BasicFunctionalityTest, TestLazyCores, DirectUpdateHandlerTest, TestBadConfig, SimplePostToolTest, TestCoreDiscovery, SignatureUpdateProcessorFactoryTest, TestExtendedDismaxParser, SpellCheckCollatorTest, SuggesterFSTTest, CoreAdminHandlerTest, SolrRequestParserTest, DocValuesTest, TestStressLucene, SpatialFilterTest, SuggesterWFSTTest, WordBreakSolrSpellCheckerTest, SolrCoreCheckLockOnStartupTest, TestAtomicUpdateErrorCases, TestWordDelimiterFilterFactory, DirectUpdateHandlerOptimizeTest, SortByFunctionTest, DistanceFunctionTest, TestSolrDeletionPolicy1, XsltUpdateRequestHandlerTest, CacheHeaderTest, IndexBasedSpellCheckerTest, TestSurroundQueryParser, LukeRequestHandlerTest, DisMaxRequestHandlerTest, TestWriterPerf, DirectSolrSpellCheckerTest, DocumentAnalysisRequestHandlerTest, TestOmitPositions, XmlUpdateRequestHandlerTest, PathHierarchyTokenizerFactoryTest, TestSolrQueryParser, SolrPluginUtilsTest, IndexSchemaTest, TestJmxIntegration, MBeansHandlerTest, QueryParsingTest, TestAnalyzedSuggestions, TestPartialUpdateDeduplication, SearchHandlerTest, HighlighterConfigTest, TestQuerySenderNoQuery, SolrIndexConfigTest, BadComponentTest, TestSolrDeletionPolicy2, MultiTermTest, TestSearchPerf, NumericFieldsTest, MinimalSchemaTest, TestElisionMultitermQuery, TestConfig, OutputWriterTest, ExternalFileFieldSortTest, TestSolrCoreProperties, TestPostingsSolrHighlighter, DirectSolrConnectionTest, NotRequiredUniqueKeyTest, TestPhraseSuggestions, TestCodecSupport, TestBM25SimilarityFactory, TimeZoneUtilsTest, TestFastOutputStream, OpenExchangeRatesOrgProviderTest, PluginInfoTest, TestSuggestSpellingConverter, DateFieldTest, ClusterStateTest, TestLRUCache, ZkNodePropsTest, SystemInfoHandlerTest, UUIDFieldTest, TestRTGBase, CursorPagingTest, CdcrReplicationDistributedZkTest, CdcrReplicationHandlerTest, CloudExitableDirectoryReaderTest, CollectionStateFormat2Test, ConcurrentDeleteAndCreateCollectionTest, ConfigSetsAPITest, CreateCollectionCleanupTest, CustomCollectionTest, DistribJoinFromCollectionTest, HttpPartitionTest]
[junit4] Completed [496/693 (2!)] on J1 in 63.49s, 1 test, 1 error <<< FAILURES!
[...truncated 63097 lines...]
[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3814
- Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3814/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC
3 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.core.TestLazyCores
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores: 1) Thread[id=9816, name=searcherExecutor-3954-thread-1, state=WAITING, group=TGRP-TestLazyCores] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores:
1) Thread[id=9816, name=searcherExecutor-3954-thread-1, state=WAITING, group=TGRP-TestLazyCores]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([59AE304D60FF3D46]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.core.TestLazyCores
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=9816, name=searcherExecutor-3954-thread-1, state=WAITING, group=TGRP-TestLazyCores] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=9816, name=searcherExecutor-3954-thread-1, state=WAITING, group=TGRP-TestLazyCores]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([59AE304D60FF3D46]:0)
FAILED: org.apache.solr.core.TestLazyCores.testNoCommit
Error Message:
Exception during query
Stack Trace:
java.lang.RuntimeException: Exception during query
at __randomizedtesting.SeedInfo.seed([59AE304D60FF3D46:86CE919CABD85EE3]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:857)
at org.apache.solr.core.TestLazyCores.check10(TestLazyCores.java:794)
at org.apache.solr.core.TestLazyCores.testNoCommit(TestLazyCores.java:776)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound='10']
xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">0</int>
<lst name="params">
<str name="q">*:*</str>
</lst>
</lst>
<result name="response" numFound="0" start="0">
</result>
</response>
request was:q=*:*
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:850)
... 41 more
Build Log:
[...truncated 11572 lines...]
[junit4] Suite: org.apache.solr.core.TestLazyCores
[junit4] 2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/init-core-data-001
[junit4] 2> 1781632 INFO (SUITE-TestLazyCores-seed#[59AE304D60FF3D46]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 1781635 INFO (SUITE-TestLazyCores-seed#[59AE304D60FF3D46]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN) w/ MAC_OS_X supressed clientAuth
[junit4] 2> 1781637 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCreateTransientFromAdmin
[junit4] 2> 1781683 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ ] o.a.s.c.SolrCores Allocating transient cache for 4 transient cores
[junit4] 2> 1781779 INFO (coreLoadExecutor-3773-thread-3) [ x:collection5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1781782 INFO (coreLoadExecutor-3773-thread-2) [ x:collection2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1781782 INFO (coreLoadExecutor-3773-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1781790 INFO (coreLoadExecutor-3773-thread-2) [ x:collection2] o.a.s.s.IndexSchema [collection2] Schema name=tiny
[junit4] 2> 1781790 INFO (coreLoadExecutor-3773-thread-3) [ x:collection5] o.a.s.s.IndexSchema [collection5] Schema name=tiny
[junit4] 2> 1781793 INFO (coreLoadExecutor-3773-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=tiny
[junit4] 2> 1781794 INFO (coreLoadExecutor-3773-thread-2) [ x:collection2] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 1781794 INFO (coreLoadExecutor-3773-thread-3) [ x:collection5] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 1781796 INFO (coreLoadExecutor-3773-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 1781798 INFO (coreLoadExecutor-3773-thread-3) [ x:collection5] o.a.s.c.CoreContainer Creating SolrCore 'collection5' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/collection5
[junit4] 2> 1781798 INFO (coreLoadExecutor-3773-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/collection1
[junit4] 2> 1781798 INFO (coreLoadExecutor-3773-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/collection1/data/]
[junit4] 2> 1781798 INFO (coreLoadExecutor-3773-thread-2) [ x:collection2] o.a.s.c.CoreContainer Creating SolrCore 'collection2' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/collection2
[junit4] 2> 1781799 INFO (coreLoadExecutor-3773-thread-3) [ x:collection5] o.a.s.c.SolrCore [[collection5] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/collection5], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/collection5/data/]
[junit4] 2> 1781799 INFO (coreLoadExecutor-3773-thread-2) [ x:collection2] o.a.s.c.SolrCore [[collection2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/collection2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/collection2/data/]
[junit4] 2> 1781803 INFO (coreLoadExecutor-3773-thread-3) [ x:collection5] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=15, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1781803 INFO (coreLoadExecutor-3773-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=15, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1781803 INFO (coreLoadExecutor-3773-thread-2) [ x:collection2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=15, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1781889 INFO (coreLoadExecutor-3773-thread-2) [ x:collection2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1781889 INFO (coreLoadExecutor-3773-thread-2) [ x:collection2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1781890 INFO (coreLoadExecutor-3773-thread-2) [ x:collection2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=10, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1781890 INFO (coreLoadExecutor-3773-thread-3) [ x:collection5] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1781890 INFO (coreLoadExecutor-3773-thread-3) [ x:collection5] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1781890 INFO (coreLoadExecutor-3773-thread-3) [ x:collection5] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=10, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1781890 INFO (coreLoadExecutor-3773-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1781890 INFO (coreLoadExecutor-3773-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1781891 INFO (coreLoadExecutor-3773-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=10, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1781892 INFO (coreLoadExecutor-3773-thread-3) [ x:collection5] o.a.s.s.SolrIndexSearcher Opening [Searcher@15c9537d[collection5] main]
[junit4] 2> 1781892 INFO (coreLoadExecutor-3773-thread-2) [ x:collection2] o.a.s.s.SolrIndexSearcher Opening [Searcher@6ef0c948[collection2] main]
[junit4] 2> 1781892 INFO (coreLoadExecutor-3773-thread-2) [ x:collection2] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/collection2/conf
[junit4] 2> 1781893 INFO (coreLoadExecutor-3773-thread-3) [ x:collection5] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/collection5/conf
[junit4] 2> 1781893 INFO (coreLoadExecutor-3773-thread-2) [ x:collection2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1781893 INFO (coreLoadExecutor-3773-thread-3) [ x:collection5] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1781893 INFO (coreLoadExecutor-3773-thread-2) [ x:collection2] o.a.s.c.SolrCores Opening transient core collection2
[junit4] 2> 1781894 INFO (searcherExecutor-3774-thread-1-processing-x:collection5) [ x:collection5] o.a.s.c.SolrCore [collection5] Registered new searcher Searcher@15c9537d[collection5] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1781894 INFO (searcherExecutor-3776-thread-1-processing-x:collection2) [ x:collection2] o.a.s.c.SolrCore [collection2] Registered new searcher Searcher@6ef0c948[collection2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1781894 INFO (coreLoadExecutor-3773-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@e60ef73[collection1] main]
[junit4] 2> 1781894 INFO (coreLoadExecutor-3773-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/collection1/conf
[junit4] 2> 1781895 INFO (coreLoadExecutor-3773-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1781895 INFO (searcherExecutor-3775-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@e60ef73[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1781918 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ ] o.a.s.h.a.CoreAdminOperation core create command loadOnStartup=true&transient=true&name=core1&action=CREATE&wt=xml
[junit4] 2> 1781934 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1781940 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.IndexSchema [core1] Schema name=tiny
[junit4] 2> 1781945 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 1781948 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.CoreContainer Creating SolrCore 'core1' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core1
[junit4] 2> 1781948 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCore [[core1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core1/data/]
[junit4] 2> 1781951 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=49, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 1782012 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1782012 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1782013 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=24, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=27.388671875, floorSegmentMB=0.953125, forceMergeDeletesPctAllowed=6.737827246473503, segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 1782013 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1d27224c[core1] main]
[junit4] 2> 1782014 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core1/conf
[junit4] 2> 1782014 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1782014 INFO (searcherExecutor-3787-thread-1-processing-x:core1) [ x:core1] o.a.s.c.SolrCore [core1] Registered new searcher Searcher@1d27224c[core1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1782015 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCores Opening transient core core1
[junit4] 2> 1782015 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.h.a.CoreAdminOperation core create command loadOnStartup=false&transient=true&name=core2&action=CREATE&wt=xml
[junit4] 2> 1782029 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1782035 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.IndexSchema [core2] Schema name=tiny
[junit4] 2> 1782048 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 1782054 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.CoreContainer Creating SolrCore 'core2' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core2
[junit4] 2> 1782055 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCore [[core2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core2/data/]
[junit4] 2> 1782059 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=683426718, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 1782126 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1782126 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1782127 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=13, maxMergeAtOnceExplicit=10, maxMergedSegmentMB=57.9892578125, floorSegmentMB=0.7041015625, forceMergeDeletesPctAllowed=29.559928710167224, segmentsPerTier=47.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.43821023908663725
[junit4] 2> 1782128 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2bc10b87[core2] main]
[junit4] 2> 1782128 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core2/conf
[junit4] 2> 1782128 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1782129 INFO (searcherExecutor-3792-thread-1-processing-x:core1) [ x:core1] o.a.s.c.SolrCore [core2] Registered new searcher Searcher@2bc10b87[core2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1782129 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCores Opening transient core core2
[junit4] 2> 1782129 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.h.a.CoreAdminOperation core create command loadOnStartup=true&transient=true&name=core3&action=CREATE&wt=xml
[junit4] 2> 1782141 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1782147 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.IndexSchema [core3] Schema name=tiny
[junit4] 2> 1782150 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 1782153 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.CoreContainer Creating SolrCore 'core3' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core3
[junit4] 2> 1782154 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCore [[core3] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core3], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core3/data/]
[junit4] 2> 1782156 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=21, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7439572040505582]
[junit4] 2> 1782224 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1782224 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1782224 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=839824161, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 1782226 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.SolrIndexSearcher Opening [Searcher@175ff952[core3] main]
[junit4] 2> 1782226 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core3/conf
[junit4] 2> 1782226 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1782227 INFO (searcherExecutor-3797-thread-1-processing-x:core1) [ x:core1] o.a.s.c.SolrCore [core3] Registered new searcher Searcher@175ff952[core3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1782227 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCores Opening transient core core3
[junit4] 2> 1782227 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.h.a.CoreAdminOperation core create command loadOnStartup=false&transient=true&name=core4&action=CREATE&wt=xml
[junit4] 2> 1782246 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1782255 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.IndexSchema [core4] Schema name=tiny
[junit4] 2> 1782260 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 1782265 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.CoreContainer Creating SolrCore 'core4' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core4
[junit4] 2> 1782266 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCore [[core4] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core4], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core4/data/]
[junit4] 2> 1782269 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=17, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.27542011496171426]
[junit4] 2> 1782343 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1782343 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1782343 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=11, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=32.9794921875, floorSegmentMB=1.771484375, forceMergeDeletesPctAllowed=1.71783767404449, segmentsPerTier=48.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5972593854452207
[junit4] 2> 1782345 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1c407b84[core4] main]
[junit4] 2> 1782345 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core4/conf
[junit4] 2> 1782346 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1782346 INFO (searcherExecutor-3802-thread-1-processing-x:core1) [ x:core1] o.a.s.c.SolrCore [core4] Registered new searcher Searcher@1c407b84[core4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1782346 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCores Opening transient core core4
[junit4] 2> 1782346 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCores Closing transient core [collection2]
[junit4] 2> 1782347 INFO (Thread-1982) [ ] o.a.s.c.SolrCore [collection2] CLOSING SolrCore org.apache.solr.core.SolrCore@6d8aa4a
[junit4] 2> 1782347 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.h.a.CoreAdminOperation core create command loadOnStartup=false&transient=true&name=core5&action=CREATE&wt=xml
[junit4] 2> 1782348 INFO (Thread-1982) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection2
[junit4] 2> 1782376 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1782382 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.IndexSchema [core5] Schema name=tiny
[junit4] 2> 1782385 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 1782389 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.CoreContainer Creating SolrCore 'core5' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core5
[junit4] 2> 1782389 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCore [[core5] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core5], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core5/data/]
[junit4] 2> 1782392 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=30, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6063172945228661]
[junit4] 2> 1782457 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1782457 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1782457 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=31, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8567274669161578]
[junit4] 2> 1782458 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.SolrIndexSearcher Opening [Searcher@122cf66c[core5] main]
[junit4] 2> 1782459 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core5/conf
[junit4] 2> 1782459 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1782460 INFO (searcherExecutor-3807-thread-1-processing-x:core1) [ x:core1] o.a.s.c.SolrCore [core5] Registered new searcher Searcher@122cf66c[core5] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1782460 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCores Opening transient core core5
[junit4] 2> 1782460 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCores Closing transient core [core1]
[junit4] 2> 1782460 INFO (Thread-1982) [ ] o.a.s.c.SolrCore [core1] CLOSING SolrCore org.apache.solr.core.SolrCore@576535f0
[junit4] 2> 1782462 INFO (Thread-1982) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.core1
[junit4] 2> 1782476 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1782482 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.IndexSchema [core1] Schema name=tiny
[junit4] 2> 1782486 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 1782489 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.CoreContainer Creating SolrCore 'core1' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core1
[junit4] 2> 1782490 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCore [[core1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core1/data/]
[junit4] 2> 1782547 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1782547 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1782549 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=22, maxMergeAtOnceExplicit=27, maxMergedSegmentMB=93.087890625, floorSegmentMB=0.236328125, forceMergeDeletesPctAllowed=20.157259956214567, segmentsPerTier=45.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.541048004126412
[junit4] 2> 1782550 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.SolrIndexSearcher Opening [Searcher@10f6ea8[core1] main]
[junit4] 2> 1782550 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core1/conf
[junit4] 2> 1782550 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1782550 INFO (searcherExecutor-3811-thread-1-processing-x:core1) [ x:core1] o.a.s.c.SolrCore [core1] Registered new searcher Searcher@10f6ea8[core1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1782551 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCores Opening transient core core1
[junit4] 2> 1782551 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCores Closing transient core [core2]
[junit4] 2> 1782551 INFO (Thread-1982) [ ] o.a.s.c.SolrCore [core2] CLOSING SolrCore org.apache.solr.core.SolrCore@d81a5b4
[junit4] 2> 1782552 INFO (Thread-1982) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.core2
[junit4] 2> 1782566 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1782582 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.IndexSchema [core2] Schema name=tiny
[junit4] 2> 1782587 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 1782593 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.CoreContainer Creating SolrCore 'core2' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core2
[junit4] 2> 1782594 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCore [[core2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core2/data/]
[junit4] 2> 1782667 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1782667 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1782668 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1483864360, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 1782670 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.SolrIndexSearcher Opening [Searcher@761b8b98[core2] main]
[junit4] 2> 1782670 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core2/conf
[junit4] 2> 1782671 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1782671 INFO (searcherExecutor-3815-thread-1-processing-x:core1) [ x:core1] o.a.s.c.SolrCore [core2] Registered new searcher Searcher@761b8b98[core2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1782671 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCores Opening transient core core2
[junit4] 2> 1782671 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCores Closing transient core [core3]
[junit4] 2> 1782671 INFO (Thread-1982) [ ] o.a.s.c.SolrCore [core3] CLOSING SolrCore org.apache.solr.core.SolrCore@2d655fe2
[junit4] 2> 1782673 INFO (Thread-1982) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.core3
[junit4] 2> 1782691 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1782697 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.IndexSchema [core3] Schema name=tiny
[junit4] 2> 1782700 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 1782704 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.CoreContainer Creating SolrCore 'core3' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core3
[junit4] 2> 1782704 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCore [[core3] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core3], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core3/data/]
[junit4] 2> 1782767 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1782767 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1782767 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=25, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.19437276754856062]
[junit4] 2> 1782768 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2471ea26[core3] main]
[junit4] 2> 1782769 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core3/conf
[junit4] 2> 1782769 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1782769 INFO (searcherExecutor-3819-thread-1-processing-x:core1) [ x:core1] o.a.s.c.SolrCore [core3] Registered new searcher Searcher@2471ea26[core3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1782770 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCores Opening transient core core3
[junit4] 2> 1782770 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCores Closing transient core [core4]
[junit4] 2> 1782770 INFO (Thread-1982) [ ] o.a.s.c.SolrCore [core4] CLOSING SolrCore org.apache.solr.core.SolrCore@4eb07a35
[junit4] 2> 1782772 INFO (Thread-1982) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.core4
[junit4] 2> 1782788 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1782794 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.IndexSchema [core4] Schema name=tiny
[junit4] 2> 1782798 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 1782801 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.CoreContainer Creating SolrCore 'core4' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core4
[junit4] 2> 1782802 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCore [[core4] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core4], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core4/data/]
[junit4] 2> 1782878 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1782878 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1782879 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=25, maxMergeAtOnceExplicit=36, maxMergedSegmentMB=48.0048828125, floorSegmentMB=1.8623046875, forceMergeDeletesPctAllowed=0.3800270929911054, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6460918199848262
[junit4] 2> 1782880 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6e701cfe[core4] main]
[junit4] 2> 1782880 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core4/conf
[junit4] 2> 1782881 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1782881 INFO (searcherExecutor-3823-thread-1-processing-x:core1) [ x:core1] o.a.s.c.SolrCore [core4] Registered new searcher Searcher@6e701cfe[core4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1782881 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCores Opening transient core core4
[junit4] 2> 1782881 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCores Closing transient core [core5]
[junit4] 2> 1782881 INFO (Thread-1982) [ ] o.a.s.c.SolrCore [core5] CLOSING SolrCore org.apache.solr.core.SolrCore@7ea332cc
[junit4] 2> 1782883 INFO (Thread-1982) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.core5
[junit4] 2> 1782908 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1782917 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.IndexSchema [core5] Schema name=tiny
[junit4] 2> 1782923 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 1782928 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.CoreContainer Creating SolrCore 'core5' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core5
[junit4] 2> 1782929 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCore [[core5] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core5], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core5/data/]
[junit4] 2> 1783013 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1783013 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1783014 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=30.1240234375, floorSegmentMB=0.3330078125, forceMergeDeletesPctAllowed=1.2539505180056332, segmentsPerTier=17.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.38804756438950394
[junit4] 2> 1783016 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.SolrIndexSearcher Opening [Searcher@18834f28[core5] main]
[junit4] 2> 1783016 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-001/core5/conf
[junit4] 2> 1783017 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1783017 INFO (searcherExecutor-3827-thread-1-processing-x:core1) [ x:core1] o.a.s.c.SolrCore [core5] Registered new searcher Searcher@18834f28[core5] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1783017 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCores Opening transient core core5
[junit4] 2> 1783017 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCores Closing transient core [core1]
[junit4] 2> 1783022 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.core2
[junit4] 2> 1783022 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCore Core core2 is not yet closed, waiting 100 ms before checking again.
[junit4] 2> 1783129 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCore Core core2 is not yet closed, waiting 100 ms before checking again.
[junit4] 2> 1783236 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCore Core core2 is not yet closed, waiting 100 ms before checking again.
[junit4] 2> 1783337 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCore Core core2 is not yet closed, waiting 100 ms before checking again.
[junit4] 2> 1783447 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCore Core core2 is not yet closed, waiting 100 ms before checking again.
[junit4] 2> 1783460 INFO (Thread-1983) [ x:core1] o.a.s.c.SolrCore [core1] CLOSING SolrCore org.apache.solr.core.SolrCore@58b66643
[junit4] 2> 1783462 INFO (Thread-1983) [ x:core1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.core1
[junit4] 2> 1783462 INFO (Thread-1983) [ x:core1] o.a.s.c.SolrCore [core2] CLOSING SolrCore org.apache.solr.core.SolrCore@4c9b5fad
[junit4] 2> 1783468 INFO (Thread-1983) [ x:core1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.core2
[junit4] 2> 1783549 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.core3
[junit4] 2> 1783549 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCore [core3] CLOSING SolrCore org.apache.solr.core.SolrCore@468b4f30
[junit4] 2> 1783550 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.core3
[junit4] 2> 1783550 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.core4
[junit4] 2> 1783550 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCore [core4] CLOSING SolrCore org.apache.solr.core.SolrCore@2c2e89c8
[junit4] 2> 1783551 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.core4
[junit4] 2> 1783551 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.core5
[junit4] 2> 1783551 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCore [core5] CLOSING SolrCore org.apache.solr.core.SolrCore@700808b
[junit4] 2> 1783551 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.core5
[junit4] 2> 1783551 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1384273406
[junit4] 2> 1783552 INFO (coreCloseExecutor-3836-thread-1-processing-x:core1) [ x:collection5] o.a.s.c.SolrCore [collection5] CLOSING SolrCore org.apache.solr.core.SolrCore@6b3341e4
[junit4] 2> 1783552 INFO (coreCloseExecutor-3836-thread-2-processing-x:core1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@46fdefb9
[junit4] 2> 1783553 INFO (coreCloseExecutor-3836-thread-1-processing-x:core1) [ x:collection5] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection5
[junit4] 2> 1783553 INFO (coreCloseExecutor-3836-thread-2-processing-x:core1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection1
[junit4] 2> 1783553 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 1783554 INFO (TEST-TestLazyCores.testCreateTransientFromAdmin-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.SolrTestCaseJ4 ###Ending testCreateTransientFromAdmin
[junit4] 2> 1783555 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.SolrTestCaseJ4 ###Starting testLazyLoad
[junit4] 2> 1783594 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCores Allocating transient cache for 4 transient cores
[junit4] 2> 1783675 INFO (coreLoadExecutor-3843-thread-2-processing-x:core1) [ x:collection2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1783675 INFO (coreLoadExecutor-3843-thread-1-processing-x:core1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1783675 INFO (coreLoadExecutor-3843-thread-3-processing-x:core1) [ x:collection5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1783693 INFO (coreLoadExecutor-3843-thread-1-processing-x:core1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=tiny
[junit4] 2> 1783693 INFO (coreLoadExecutor-3843-thread-3-processing-x:core1) [ x:collection5] o.a.s.s.IndexSchema [collection5] Schema name=tiny
[junit4] 2> 1783694 INFO (coreLoadExecutor-3843-thread-2-processing-x:core1) [ x:collection2] o.a.s.s.IndexSchema [collection2] Schema name=tiny
[junit4] 2> 1783698 INFO (coreLoadExecutor-3843-thread-2-processing-x:core1) [ x:collection2] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 1783699 INFO (coreLoadExecutor-3843-thread-3-processing-x:core1) [ x:collection5] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 1783700 INFO (coreLoadExecutor-3843-thread-1-processing-x:core1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 1783705 INFO (coreLoadExecutor-3843-thread-1-processing-x:core1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-002/collection1
[junit4] 2> 1783705 INFO (coreLoadExecutor-3843-thread-2-processing-x:core1) [ x:collection2] o.a.s.c.CoreContainer Creating SolrCore 'collection2' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-002/collection2
[junit4] 2> 1783705 INFO (coreLoadExecutor-3843-thread-3-processing-x:core1) [ x:collection5] o.a.s.c.CoreContainer Creating SolrCore 'collection5' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-002/collection5
[junit4] 2> 1783706 INFO (coreLoadExecutor-3843-thread-3-processing-x:core1) [ x:collection5] o.a.s.c.SolrCore [[collection5] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-002/collection5], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-002/collection5/data/]
[junit4] 2> 1783706 INFO (coreLoadExecutor-3843-thread-2-processing-x:core1) [ x:collection2] o.a.s.c.SolrCore [[collection2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-002/collection2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-002/collection2/data/]
[junit4] 2> 1783706 INFO (coreLoadExecutor-3843-thread-1-processing-x:core1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-002/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-002/collection1/data/]
[junit4] 2> 1783709 INFO (coreLoadExecutor-3843-thread-3-processing-x:core1) [ x:collection5] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=15, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1783710 INFO (coreLoadExecutor-3843-thread-1-processing-x:core1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=15, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1783710 INFO (coreLoadExecutor-3843-thread-2-processing-x:core1) [ x:collection2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=15, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1783805 INFO (coreLoadExecutor-3843-thread-1-processing-x:core1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1783805 INFO (coreLoadExecutor-3843-thread-1-processing-x:core1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1783805 INFO (coreLoadExecutor-3843-thread-3-processing-x:core1) [ x:collection5] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1783805 INFO (coreLoadExecutor-3843-thread-1-processing-x:core1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=10, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1783805 INFO (coreLoadExecutor-3843-thread-3-processing-x:core1) [ x:collection5] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1783806 INFO (coreLoadExecutor-3843-thread-2-processing-x:core1) [ x:collection2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1783806 INFO (coreLoadExecutor-3843-thread-3-processing-x:core1) [ x:collection5] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=10, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1783806 INFO (coreLoadExecutor-3843-thread-2-processing-x:core1) [ x:collection2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1783807 INFO (coreLoadExecutor-3843-thread-1-processing-x:core1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@20b71828[collection1] main]
[junit4] 2> 1783807 INFO (coreLoadExecutor-3843-thread-3-processing-x:core1) [ x:collection5] o.a.s.s.SolrIndexSearcher Opening [Searcher@4601709f[collection5] main]
[junit4] 2> 1783807 INFO (coreLoadExecutor-3843-thread-1-processing-x:core1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-002/collection1/conf
[junit4] 2> 1783808 INFO (coreLoadExecutor-3843-thread-3-processing-x:core1) [ x:collection5] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-002/collection5/conf
[junit4] 2> 1783808 INFO (coreLoadExecutor-3843-thread-1-processing-x:core1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1783808 INFO (coreLoadExecutor-3843-thread-3-processing-x:core1) [ x:collection5] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1783808 INFO (searcherExecutor-3844-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@20b71828[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1783808 INFO (searcherExecutor-3846-thread-1-processing-x:collection5) [ x:collection5] o.a.s.c.SolrCore [collection5] Registered new searcher Searcher@4601709f[collection5] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1783808 INFO (coreLoadExecutor-3843-thread-2-processing-x:core1) [ x:collection2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=10, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1783809 INFO (coreLoadExecutor-3843-thread-2-processing-x:core1) [ x:collection2] o.a.s.s.SolrIndexSearcher Opening [Searcher@244a7d99[collection2] main]
[junit4] 2> 1783810 INFO (coreLoadExecutor-3843-thread-2-processing-x:core1) [ x:collection2] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-002/collection2/conf
[junit4] 2> 1783810 INFO (coreLoadExecutor-3843-thread-2-processing-x:core1) [ x:collection2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1783810 INFO (searcherExecutor-3845-thread-1-processing-x:collection2) [ x:collection2] o.a.s.c.SolrCore [collection2] Registered new searcher Searcher@244a7d99[collection2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1783810 INFO (coreLoadExecutor-3843-thread-2-processing-x:core1) [ x:collection2] o.a.s.c.SolrCores Opening transient core collection2
[junit4] 2> 1783832 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1783847 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.IndexSchema [collection3] Schema name=tiny
[junit4] 2> 1783853 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 1783858 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.CoreContainer Creating SolrCore 'collection3' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-002/collection3
[junit4] 2> 1783859 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCore [[collection3] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-002/collection3], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-002/collection3/data/]
[junit4] 2> 1783864 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=37, maxMergeAtOnceExplicit=17, maxMergedSegmentMB=83.2529296875, floorSegmentMB=2.1865234375, forceMergeDeletesPctAllowed=0.36776241474489324, segmentsPerTier=50.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1566808019931316
[junit4] 2> 1783957 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1783957 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1783958 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=21, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1783959 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5d73492[collection3] main]
[junit4] 2> 1783959 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-002/collection3/conf
[junit4] 2> 1783960 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1783960 INFO (searcherExecutor-3856-thread-1-processing-x:core1) [ x:core1] o.a.s.c.SolrCore [collection3] Registered new searcher Searcher@5d73492[collection3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1783960 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCores Opening transient core collection3
[junit4] 2> 1783979 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1783983 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.IndexSchema [collection4] Schema name=tiny
[junit4] 2> 1783991 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 1783995 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.CoreContainer Creating SolrCore 'collection4' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-002/collection4
[junit4] 2> 1783995 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCore [[collection4] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-002/collection4], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-002/collection4/data/]
[junit4] 2> 1783998 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=30, maxMergeAtOnceExplicit=36, maxMergedSegmentMB=87.390625, floorSegmentMB=0.541015625, forceMergeDeletesPctAllowed=12.922545561063348, segmentsPerTier=14.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.30532050574314795
[junit4] 2> 1784084 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1784084 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1784084 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=44, maxMergedSegmentMB=1.5078125, floorSegmentMB=1.5869140625, forceMergeDeletesPctAllowed=29.466776509892657, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8394521596094265
[junit4] 2> 1784085 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2a84f7e2[collection4] main]
[junit4] 2> 1784085 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-002/collection4/conf
[junit4] 2> 1784086 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1784086 INFO (searcherExecutor-3860-thread-1-processing-x:core1) [ x:core1] o.a.s.c.SolrCore [collection4] Registered new searcher Searcher@2a84f7e2[collection4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1784086 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.CoreContainer Shutting down CoreContainer instance=613482775
[junit4] 2> 1784087 INFO (coreCloseExecutor-3864-thread-1-processing-x:core1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@76c85303
[junit4] 2> 1784087 INFO (coreCloseExecutor-3864-thread-2-processing-x:core1) [ x:collection5] o.a.s.c.SolrCore [collection5] CLOSING SolrCore org.apache.solr.core.SolrCore@7f1ff4f6
[junit4] 2> 1784089 INFO (coreCloseExecutor-3864-thread-1-processing-x:core1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection1
[junit4] 2> 1784089 INFO (coreCloseExecutor-3864-thread-2-processing-x:core1) [ x:collection5] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection5
[junit4] 2> 1784090 INFO (coreCloseExecutor-3864-thread-3-processing-x:core1) [ x:collection4] o.a.s.c.SolrCore [collection4] CLOSING SolrCore org.apache.solr.core.SolrCore@528fb28d
[junit4] 2> 1784091 INFO (coreCloseExecutor-3864-thread-4-processing-x:core1) [ x:collection2] o.a.s.c.SolrCore [collection2] CLOSING SolrCore org.apache.solr.core.SolrCore@72114d8
[junit4] 2> 1784092 INFO (coreCloseExecutor-3864-thread-3-processing-x:core1) [ x:collection4] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection4
[junit4] 2> 1784093 INFO (coreCloseExecutor-3864-thread-4-processing-x:core1) [ x:collection2] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection2
[junit4] 2> 1784093 INFO (coreCloseExecutor-3864-thread-5-processing-x:core1) [ x:collection3] o.a.s.c.SolrCore [collection3] CLOSING SolrCore org.apache.solr.core.SolrCore@e09a446
[junit4] 2> 1784095 INFO (coreCloseExecutor-3864-thread-5-processing-x:core1) [ x:collection3] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection3
[junit4] 2> 1784095 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 1784095 INFO (TEST-TestLazyCores.testLazyLoad-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.SolrTestCaseJ4 ###Ending testLazyLoad
[junit4] 2> 1784096 INFO (TEST-TestLazyCores.testMidUseUnload-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.SolrTestCaseJ4 ###Starting testMidUseUnload
[junit4] 2> 1784134 INFO (TEST-TestLazyCores.testMidUseUnload-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.c.SolrCores Allocating transient cache for 4 transient cores
[junit4] 2> 1784212 INFO (coreLoadExecutor-3871-thread-1-processing-x:core1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1784212 INFO (coreLoadExecutor-3871-thread-2-processing-x:core1) [ x:collection2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1784213 INFO (coreLoadExecutor-3871-thread-3-processing-x:core1) [ x:collection5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1784223 INFO (coreLoadExecutor-3871-thread-1-processing-x:core1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=tiny
[junit4] 2> 1784223 INFO (coreLoadExecutor-3871-thread-2-processing-x:core1) [ x:collection2] o.a.s.s.IndexSchema [collection2] Schema name=tiny
[junit4] 2> 1784224 INFO (coreLoadExecutor-3871-thread-3-processing-x:core1) [ x:collection5] o.a.s.s.IndexSchema [collection5] Schema name=tiny
[junit4] 2> 1784229 INFO (coreLoadExecutor-3871-thread-3-processing-x:core1) [ x:collection5] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 1784229 INFO (coreLoadExecutor-3871-thread-2-processing-x:core1) [ x:collection2] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 1784229 INFO (coreLoadExecutor-3871-thread-1-processing-x:core1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 1784234 INFO (coreLoadExecutor-3871-thread-1-processing-x:core1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-003/collection1
[junit4] 2> 1784235 INFO (coreLoadExecutor-3871-thread-2-processing-x:core1) [ x:collection2] o.a.s.c.CoreContainer Creating SolrCore 'collection2' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-003/collection2
[junit4] 2> 1784235 INFO (coreLoadExecutor-3871-thread-2-processing-x:core1) [ x:collection2] o.a.s.c.SolrCore [[collection2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-003/collection2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-003/collection2/data/]
[junit4] 2> 1784235 INFO (coreLoadExecutor-3871-thread-3-processing-x:core1) [ x:collection5] o.a.s.c.CoreContainer Creating SolrCore 'collection5' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-003/collection5
[junit4] 2> 1784235 INFO (coreLoadExecutor-3871-thread-1-processing-x:core1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-003/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-003/collection1/data/]
[junit4] 2> 1784235 INFO (coreLoadExecutor-3871-thread-3-processing-x:core1) [ x:collection5] o.a.s.c.SolrCore [[collection5] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-003/collection5], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001/tempDir-003/collection5/data/]
[junit4] 2> 1784239 INFO (coreLoadExecutor-3871-thread-1-processing-x:core1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=15, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1784240 INFO (coreLoadExecutor-3871-thread-3-processing-x:core1) [ x:collection5] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=15, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1784241 INFO (coreLoadExecutor-3871-thread-2-processing-x:core1) [ x:collection2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=15, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1784350 INFO (coreLoadExecutor-3871-thread-3-processing-x:core1) [ x:collection5] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1784350 INFO (coreLoadExecutor-3871-thread-3-processing-x:core1) [ x:collection5] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1784351 INFO (coreLoadExecutor-3871-thread-3-processing-x:core1) [ x:collection5] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=10, maxMergeSize=2147483648, maxMergeS
[...truncated too long message...]
:core1) [ x:collection2] o.a.s.c.SolrCore [collection2] CLOSING SolrCore org.apache.solr.core.SolrCore@1877d843
[junit4] 2> 1788339 INFO (coreCloseExecutor-4094-thread-1-processing-x:core1) [ x:collection5] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection5
[junit4] 2> 1788341 INFO (coreCloseExecutor-4094-thread-4-processing-x:core1) [ x:collection2] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection2
[junit4] 2> 1788341 INFO (coreCloseExecutor-4094-thread-2-processing-x:core1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection1
[junit4] 2> 1788342 INFO (coreCloseExecutor-4094-thread-3-processing-x:core1) [ x:collection4] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection4
[junit4] 2> 1788343 INFO (TEST-TestLazyCores.testLazySearch-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 1788343 INFO (TEST-TestLazyCores.testLazySearch-seed#[59AE304D60FF3D46]) [ x:core1] o.a.s.SolrTestCaseJ4 ###Ending testLazySearch
[junit4] 2> 1788343 INFO (SUITE-TestLazyCores-seed#[59AE304D60FF3D46]-worker) [ x:core1] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> 1788343 INFO (SUITE-TestLazyCores-seed#[59AE304D60FF3D46]-worker) [ x:core1] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1868775534
[junit4] 2> 1788343 INFO (SUITE-TestLazyCores-seed#[59AE304D60FF3D46]-worker) [ x:core1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_59AE304D60FF3D46-001
[junit4] 2> Feb 02, 2017 7:00:17 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> Feb 02, 2017 7:00:37 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores:
[junit4] 2> 1) Thread[id=9816, name=searcherExecutor-3954-thread-1, state=WAITING, group=TGRP-TestLazyCores]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> Feb 02, 2017 7:00:37 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=9816, name=searcherExecutor-3954-thread-1, state=WAITING, group=TGRP-TestLazyCores]
[junit4] 2> Feb 02, 2017 7:00:40 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=9816, name=searcherExecutor-3954-thread-1, state=WAITING, group=TGRP-TestLazyCores]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> NOTE: test params are: codec=Lucene70, sim=RandomSimilarity(queryNorm=false): {}, locale=ms-MY, timezone=America/St_Lucia
[junit4] 2> NOTE: Mac OS X 10.11.6 x86_64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=3,threads=2,free=152006432,total=422576128
[junit4] 2> NOTE: All tests run in this JVM: [TestLeaderInitiatedRecoveryThread, TolerantUpdateProcessorTest, TestFieldResource, TestSolrCloudSnapshots, TestCryptoKeys, TestDocBasedVersionConstraints, TestSurroundQueryParser, TestMacroExpander, TestRuleBasedAuthorizationPlugin, TestRestoreCore, HdfsDirectoryTest, ZkNodePropsTest, MoreLikeThisHandlerTest, DistribJoinFromCollectionTest, TestReRankQParserPlugin, TestSolrFieldCacheMBean, TestCharFilters, TestFieldCacheReopen, SyncSliceTest, TestSimpleQParserPlugin, FileUtilsTest, DistributedFacetPivotWhiteBoxTest, TestDistributedSearch, HLLSerializationTest, CdcrReplicationHandlerTest, TestDelegationWithHadoopAuth, CdcrBootstrapTest, DOMUtilTest, TestFieldCacheVsDocValues, TestTolerantSearch, TestElisionMultitermQuery, TestPseudoReturnFields, PingRequestHandlerTest, LeaderFailureAfterFreshStartTest, SystemInfoHandlerTest, TestJsonRequest, TestLRUCache, TestSolrCLIRunExample, PreAnalyzedUpdateProcessorTest, UnloadDistributedZkTest, FacetPivotSmallTest, TestStressLucene, TestBulkSchemaConcurrent, TestStressUserVersions, LeaderElectionTest, TestDefaultStatsCache, BlockDirectoryTest, SolrCoreMetricManagerTest, DistributedQueryComponentOptimizationTest, TestLockTree, TestDistributedStatsComponentCardinality, TestCoreAdminApis, MergeStrategyTest, SecurityConfHandlerTest, TestManagedSynonymFilterFactory, SimpleFacetsTest, ConfigSetsAPITest, SolrCoreTest, TestCorePropertiesReload, TestSolrJ, DisMaxRequestHandlerTest, TriLevelCompositeIdRoutingTest, TestReplicaProperties, TestTolerantUpdateProcessorRandomCloud, TestSizeLimitedDistributedMap, SolrCLIZkUtilsTest, XmlUpdateRequestHandlerTest, TestSolrCloudWithKerberosAlt, OverseerTaskQueueTest, SparseHLLTest, TestCSVResponseWriter, TestCollectionAPI, TestXIncludeConfig, RulesTest, TestOnReconnectListenerSupport, ReplaceNodeTest, TestNRTOpen, DistributedMLTComponentTest, TestPerFieldSimilarity, TestUniqueKeyFieldResource, TemplateUpdateProcessorTest, TestNumericTerms64, TestLFUCache, DistribCursorPagingTest, ForceLeaderTest, LeaderInitiatedRecoveryOnShardRestartTest, TestHashQParserPlugin, TestSolrCloudWithHadoopAuthPlugin, TimeZoneUtilsTest, DocumentBuilderTest, BinaryUpdateRequestHandlerTest, ShardRoutingTest, TestCloudRecovery, TestSortByMinMaxFunction, TestPostingsSolrHighlighter, HdfsThreadLeakTest, ConnectionReuseTest, TestQuerySenderNoQuery, CollectionReloadTest, FastVectorHighlighterTest, StatelessScriptUpdateProcessorFactoryTest, AsyncCallRequestStatusResponseTest, TestBinaryResponseWriter, TestLazyCores]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestLazyCores -Dtests.seed=59AE304D60FF3D46 -Dtests.slow=true -Dtests.locale=ms-MY -Dtests.timezone=America/St_Lucia -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | TestLazyCores (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores:
[junit4] > 1) Thread[id=9816, name=searcherExecutor-3954-thread-1, state=WAITING, group=TGRP-TestLazyCores]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > at __randomizedtesting.SeedInfo.seed([59AE304D60FF3D46]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=9816, name=searcherExecutor-3954-thread-1, state=WAITING, group=TGRP-TestLazyCores]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > at __randomizedtesting.SeedInfo.seed([59AE304D60FF3D46]:0)
[junit4] Completed [281/692 (1!)] on J0 in 29.77s, 9 tests, 3 errors <<< FAILURES!
[...truncated 63762 lines...]
[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3813
- Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3813/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC
5 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.RecoveryZkTest
Error Message:
Error starting up MiniSolrCloudCluster
Stack Trace:
java.lang.Exception: Error starting up MiniSolrCloudCluster
at __randomizedtesting.SeedInfo.seed([3B776154A0F4835F]:0)
at org.apache.solr.cloud.MiniSolrCloudCluster.checkForExceptions(MiniSolrCloudCluster.java:493)
at org.apache.solr.cloud.MiniSolrCloudCluster.<init>(MiniSolrCloudCluster.java:243)
at org.apache.solr.cloud.SolrCloudTestCase$Builder.configure(SolrCloudTestCase.java:188)
at org.apache.solr.cloud.RecoveryZkTest.setupCluster(RecoveryZkTest.java:47)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:847)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Suppressed: java.io.IOException: Too many open files
at sun.nio.ch.IOUtil.makePipe(Native Method)
at sun.nio.ch.KQueueSelectorImpl.<init>(KQueueSelectorImpl.java:84)
at sun.nio.ch.KQueueSelectorProvider.openSelector(KQueueSelectorProvider.java:42)
at java.nio.channels.Selector.open(Selector.java:227)
at org.eclipse.jetty.io.ManagedSelector.newSelector(ManagedSelector.java:96)
at org.eclipse.jetty.io.ManagedSelector.doStart(ManagedSelector.java:90)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:113)
at org.eclipse.jetty.io.SelectorManager.doStart(SelectorManager.java:273)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105)
at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:268)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:81)
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:235)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:401)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376)
at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.RecoveryZkTest
Error Message:
9 threads leaked from SUITE scope at org.apache.solr.cloud.RecoveryZkTest: 1) Thread[id=15913, name=qtp1977136872-15913, state=TIMED_WAITING, group=TGRP-RecoveryZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 2) Thread[id=15925, name=qtp1977136872-15925, state=TIMED_WAITING, group=TGRP-RecoveryZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 3) Thread[id=15915, name=qtp1977136872-15915, state=TIMED_WAITING, group=TGRP-RecoveryZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 4) Thread[id=15926, name=qtp1977136872-15926, state=TIMED_WAITING, group=TGRP-RecoveryZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 5) Thread[id=15928, name=org.eclipse.jetty.server.session.HashSessionManager@a25907Timer, state=TIMED_WAITING, group=TGRP-RecoveryZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 6) Thread[id=15927, name=qtp1977136872-15927, state=TIMED_WAITING, group=TGRP-RecoveryZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 7) Thread[id=15918, name=qtp1977136872-15918, state=TIMED_WAITING, group=TGRP-RecoveryZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 8) Thread[id=15923, name=qtp1977136872-15923, state=TIMED_WAITING, group=TGRP-RecoveryZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 9) Thread[id=15912, name=qtp1977136872-15912, state=TIMED_WAITING, group=TGRP-RecoveryZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 9 threads leaked from SUITE scope at org.apache.solr.cloud.RecoveryZkTest:
1) Thread[id=15913, name=qtp1977136872-15913, state=TIMED_WAITING, group=TGRP-RecoveryZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
2) Thread[id=15925, name=qtp1977136872-15925, state=TIMED_WAITING, group=TGRP-RecoveryZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
3) Thread[id=15915, name=qtp1977136872-15915, state=TIMED_WAITING, group=TGRP-RecoveryZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
4) Thread[id=15926, name=qtp1977136872-15926, state=TIMED_WAITING, group=TGRP-RecoveryZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
5) Thread[id=15928, name=org.eclipse.jetty.server.session.HashSessionManager@a25907Timer, state=TIMED_WAITING, group=TGRP-RecoveryZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
6) Thread[id=15927, name=qtp1977136872-15927, state=TIMED_WAITING, group=TGRP-RecoveryZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
7) Thread[id=15918, name=qtp1977136872-15918, state=TIMED_WAITING, group=TGRP-RecoveryZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
8) Thread[id=15923, name=qtp1977136872-15923, state=TIMED_WAITING, group=TGRP-RecoveryZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
9) Thread[id=15912, name=qtp1977136872-15912, state=TIMED_WAITING, group=TGRP-RecoveryZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([3B776154A0F4835F]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.RecoveryZkTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=15928, name=org.eclipse.jetty.server.session.HashSessionManager@a25907Timer, state=TIMED_WAITING, group=TGRP-RecoveryZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=15928, name=org.eclipse.jetty.server.session.HashSessionManager@a25907Timer, state=TIMED_WAITING, group=TGRP-RecoveryZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([3B776154A0F4835F]:0)
FAILED: org.apache.solr.cloud.ShardRoutingTest.test
Error Message:
java.io.IOException: Couldn't instantiate org.apache.zookeeper.ClientCnxnSocketNIO
Stack Trace:
org.apache.solr.common.SolrException: java.io.IOException: Couldn't instantiate org.apache.zookeeper.ClientCnxnSocketNIO
at __randomizedtesting.SeedInfo.seed([3B776154A0F4835F:B3235E8E0E08EEA7]:0)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:170)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:116)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:106)
at org.apache.solr.common.cloud.ZkStateReader.<init>(ZkStateReader.java:226)
at org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:121)
at org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:615)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.initCloud(AbstractFullDistribZkTestBase.java:268)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:330)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:983)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Couldn't instantiate org.apache.zookeeper.ClientCnxnSocketNIO
at org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:1782)
at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:447)
at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:380)
at org.apache.solr.common.cloud.SolrZooKeeper.<init>(SolrZooKeeper.java:43)
at org.apache.solr.common.cloud.ZkClientConnectionStrategy.createSolrZooKeeper(ZkClientConnectionStrategy.java:105)
at org.apache.solr.common.cloud.DefaultConnectionStrategy.connect(DefaultConnectionStrategy.java:37)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:145)
... 40 more
Caused by: java.io.IOException: Too many open files
at sun.nio.ch.IOUtil.makePipe(Native Method)
at sun.nio.ch.KQueueSelectorImpl.<init>(KQueueSelectorImpl.java:84)
at sun.nio.ch.KQueueSelectorProvider.openSelector(KQueueSelectorProvider.java:42)
at java.nio.channels.Selector.open(Selector.java:227)
at org.apache.zookeeper.ClientCnxnSocketNIO.<init>(ClientCnxnSocketNIO.java:43)
at sun.reflect.GeneratedConstructorAccessor4.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:1779)
... 46 more
FAILED: org.apache.solr.core.OpenCloseCoreStressTest.test5Seconds
Error Message:
Core 00000_core bad! expected:<5> but was:<4>
Stack Trace:
java.lang.AssertionError: Core 00000_core bad! expected:<5> but was:<4>
at __randomizedtesting.SeedInfo.seed([3B776154A0F4835F:DD583111C19A25CB]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.core.OpenCloseCoreStressTest.checkResults(OpenCloseCoreStressTest.java:283)
at org.apache.solr.core.OpenCloseCoreStressTest.doStress(OpenCloseCoreStressTest.java:182)
at org.apache.solr.core.OpenCloseCoreStressTest.test5Seconds(OpenCloseCoreStressTest.java:110)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11768 lines...]
[junit4] Suite: org.apache.solr.core.OpenCloseCoreStressTest
[junit4] 2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_3B776154A0F4835F-001/init-core-data-001
[junit4] 2> 2007727 INFO (SUITE-OpenCloseCoreStressTest-seed#[3B776154A0F4835F]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields
[junit4] 2> 2007731 INFO (SUITE-OpenCloseCoreStressTest-seed#[3B776154A0F4835F]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN) w/ MAC_OS_X supressed clientAuth
[junit4] IGNOR/A 0.00s J1 | OpenCloseCoreStressTest.test1Hour
[junit4] > Assumption #1: 'weekly' test group is disabled (@Weekly())
[junit4] IGNOR/A 0.00s J1 | OpenCloseCoreStressTest.test15Seconds
[junit4] > Assumption #1: 'nightly' test group is disabled (@Nightly())
[junit4] 2> 2007733 INFO (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[3B776154A0F4835F]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test5Seconds
[junit4] 2> 2007762 INFO (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[3B776154A0F4835F]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2007764 INFO (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[3B776154A0F4835F]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5d5cdfa6{/solr,null,AVAILABLE}
[junit4] 2> 2007767 INFO (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[3B776154A0F4835F]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@45fdf408{HTTP/1.1,[http/1.1]}{127.0.0.1:58354}
[junit4] 2> 2007767 INFO (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[3B776154A0F4835F]) [ ] o.e.j.s.Server Started @2012746ms
[junit4] 2> 2007767 INFO (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[3B776154A0F4835F]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=58354}
[junit4] 2> 2007767 ERROR (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[3B776154A0F4835F]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2007767 INFO (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[3B776154A0F4835F]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 2007767 INFO (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[3B776154A0F4835F]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 2007767 INFO (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[3B776154A0F4835F]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2007768 INFO (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[3B776154A0F4835F]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-02T09:40:22.549Z
[junit4] 2> 2007768 INFO (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[3B776154A0F4835F]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_3B776154A0F4835F-001/tempDir-001/solr.xml
[junit4] 2> 2007846 INFO (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[3B776154A0F4835F]) [ ] o.a.s.c.CorePropertiesLocator Found 5 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_3B776154A0F4835F-001/tempDir-001
[junit4] 2> 2007846 INFO (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[3B776154A0F4835F]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [00000_core, 00001_core, 00002_core, 00003_core, 00004_core]
[junit4] 2> 2008477 INFO (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[3B776154A0F4835F]) [ ] o.a.s.c.OpenCloseCoreStressTest Starting indexing and querying
[junit4] 2> 2008477 INFO (TEST-OpenCloseCoreStressTest.test5Seconds-seed#[3B776154A0F4835F]) [ ] o.a.s.c.OpenCloseCoreStressTest
[junit4] 2>
[junit4] 2>
[junit4] 2>
[junit4] 2>
[junit4] 2> Starting a 5 second cycle, seconds left: 5. Seconds run so far: 0.
[junit4] 2> 2008486 INFO (Thread-2926) [ ] o.a.s.c.OneIndexer Starting indexing thread: 15827
[junit4] 2> 2008486 INFO (Thread-2924) [ ] o.a.s.c.OneIndexer Starting indexing thread: 15825
[junit4] 2> 2008486 INFO (Thread-2925) [ ] o.a.s.c.OneIndexer Starting indexing thread: 15826
[junit4] 2> 2008486 INFO (Thread-2927) [ ] o.a.s.c.OneIndexer Starting indexing thread: 15828
[junit4] 2> 2008487 INFO (Thread-2928) [ ] o.a.s.c.OneIndexer Starting indexing thread: 15829
[junit4] 2> 2008504 INFO (Thread-2929) [ ] o.a.s.c.OneQuery Starting query thread: 15831
[junit4] 2> 2008506 INFO (Thread-2930) [ ] o.a.s.c.OneQuery Starting query thread: 15832
[junit4] 2> 2008509 WARN (qtp2131987516-15808) [ x:00000_core] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 2008510 WARN (qtp2131987516-15807) [ x:00004_core] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 2008510 INFO (qtp2131987516-15808) [ x:00000_core] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2008510 WARN (qtp2131987516-15805) [ x:00002_core] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 2008511 INFO (qtp2131987516-15807) [ x:00004_core] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2008511 INFO (qtp2131987516-15805) [ x:00002_core] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2008521 INFO (Thread-2931) [ ] o.a.s.c.OneQuery Starting query thread: 15833
[junit4] 2> 2008524 INFO (Thread-2933) [ ] o.a.s.c.OneQuery Starting query thread: 15835
[junit4] 2> 2008524 INFO (Thread-2932) [ ] o.a.s.c.OneQuery Starting query thread: 15834
[junit4] 2> 2008525 INFO (qtp2131987516-15807) [ x:00004_core] o.a.s.s.IndexSchema [00004_core] Schema name=tiny
[junit4] 2> 2008525 INFO (qtp2131987516-15808) [ x:00000_core] o.a.s.s.IndexSchema [00000_core] Schema name=tiny
[junit4] 2> 2008525 INFO (qtp2131987516-15805) [ x:00002_core] o.a.s.s.IndexSchema [00002_core] Schema name=tiny
[junit4] 2> 2008529 INFO (qtp2131987516-15807) [ x:00004_core] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 2008529 INFO (qtp2131987516-15808) [ x:00000_core] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 2008529 INFO (qtp2131987516-15805) [ x:00002_core] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 2008533 INFO (qtp2131987516-15807) [ x:00004_core] o.a.s.c.CoreContainer Creating SolrCore '00004_core' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_3B776154A0F4835F-001/tempDir-001/00004_core
[junit4] 2> 2008534 INFO (qtp2131987516-15805) [ x:00002_core] o.a.s.c.CoreContainer Creating SolrCore '00002_core' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_3B776154A0F4835F-001/tempDir-001/00002_core
[junit4] 2> 2008534 INFO (qtp2131987516-15807) [ x:00004_core] o.a.s.c.SolrCore [[00004_core] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_3B776154A0F4835F-001/tempDir-001/00004_core], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_3B776154A0F4835F-001/tempDir-001/00004_core/data/]
[junit4] 2> 2008534 INFO (qtp2131987516-15805) [ x:00002_core] o.a.s.c.SolrCore [[00002_core] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_3B776154A0F4835F-001/tempDir-001/00002_core], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_3B776154A0F4835F-001/tempDir-001/00002_core/data/]
[junit4] 2> 2008534 INFO (qtp2131987516-15808) [ x:00000_core] o.a.s.c.CoreContainer Creating SolrCore '00000_core' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_3B776154A0F4835F-001/tempDir-001/00000_core
[junit4] 2> 2008535 INFO (qtp2131987516-15808) [ x:00000_core] o.a.s.c.SolrCore [[00000_core] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_3B776154A0F4835F-001/tempDir-001/00000_core], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_3B776154A0F4835F-001/tempDir-001/00000_core/data/]
[junit4] 2> 2008536 INFO (qtp2131987516-15807) [ x:00004_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=39, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.478960634184331]
[junit4] 2> 2008536 INFO (qtp2131987516-15805) [ x:00002_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=39, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.478960634184331]
[junit4] 2> 2008536 INFO (qtp2131987516-15808) [ x:00000_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=39, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.478960634184331]
[junit4] 2> 2008613 INFO (qtp2131987516-15805) [ x:00002_core] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2008613 INFO (qtp2131987516-15805) [ x:00002_core] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2008613 INFO (qtp2131987516-15807) [ x:00004_core] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2008613 INFO (qtp2131987516-15807) [ x:00004_core] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2008614 INFO (qtp2131987516-15805) [ x:00002_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=6, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5949200062896619]
[junit4] 2> 2008614 INFO (qtp2131987516-15808) [ x:00000_core] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2008615 INFO (qtp2131987516-15808) [ x:00000_core] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2008614 INFO (qtp2131987516-15807) [ x:00004_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=6, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5949200062896619]
[junit4] 2> 2008615 INFO (qtp2131987516-15808) [ x:00000_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=6, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5949200062896619]
[junit4] 2> 2008615 INFO (qtp2131987516-15805) [ x:00002_core] o.a.s.s.SolrIndexSearcher Opening [Searcher@14d81032[00002_core] main]
[junit4] 2> 2008615 INFO (qtp2131987516-15807) [ x:00004_core] o.a.s.s.SolrIndexSearcher Opening [Searcher@60de80c2[00004_core] main]
[junit4] 2> 2008616 INFO (qtp2131987516-15805) [ x:00002_core] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_3B776154A0F4835F-001/tempDir-001/00002_core/conf
[junit4] 2> 2008616 INFO (qtp2131987516-15807) [ x:00004_core] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_3B776154A0F4835F-001/tempDir-001/00004_core/conf
[junit4] 2> 2008616 INFO (qtp2131987516-15805) [ x:00002_core] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2008616 INFO (qtp2131987516-15807) [ x:00004_core] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2008616 INFO (qtp2131987516-15807) [ x:00004_core] o.a.s.c.SolrCores Opening transient core 00004_core
[junit4] 2> 2008616 INFO (searcherExecutor-7022-thread-1-processing-x:00002_core) [ x:00002_core] o.a.s.c.SolrCore [00002_core] Registered new searcher Searcher@14d81032[00002_core] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2008616 INFO (searcherExecutor-7021-thread-1-processing-x:00004_core) [ x:00004_core] o.a.s.c.SolrCore [00004_core] Registered new searcher Searcher@60de80c2[00004_core] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2008617 INFO (qtp2131987516-15805) [ x:00002_core] o.a.s.c.SolrCores Opening transient core 00002_core
[junit4] 2> 2008617 INFO (qtp2131987516-15802) [ x:00004_core] o.a.s.c.S.Request [00004_core] webapp=/solr path=/select params={q=*:*&qt=/select&wt=javabin&version=2} hits=0 status=0 QTime=3
[junit4] 2> 2008617 INFO (qtp2131987516-15808) [ x:00000_core] o.a.s.s.SolrIndexSearcher Opening [Searcher@21ff50a8[00000_core] main]
[junit4] 2> 2008618 INFO (qtp2131987516-15808) [ x:00000_core] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_3B776154A0F4835F-001/tempDir-001/00000_core/conf
[junit4] 2> 2008618 INFO (qtp2131987516-15808) [ x:00000_core] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2008618 INFO (qtp2131987516-15807) [ x:00004_core] o.a.s.u.p.LogUpdateProcessorFactory [00004_core] webapp=/solr path=/update params={wt=javabin&version=2}{add=[id3]} 0 128
[junit4] 2> 2008618 INFO (searcherExecutor-7023-thread-1-processing-x:00000_core) [ x:00000_core] o.a.s.c.SolrCore [00000_core] Registered new searcher Searcher@21ff50a8[00000_core] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2008618 INFO (qtp2131987516-15808) [ x:00000_core] o.a.s.c.SolrCores Opening transient core 00000_core
[junit4] 2> 2008619 INFO (qtp2131987516-15805) [ x:00002_core] o.a.s.u.p.LogUpdateProcessorFactory [00002_core] webapp=/solr path=/update params={wt=javabin&version=2}{add=[id2]} 0 129
[junit4] 2> 2008620 INFO (qtp2131987516-15805) [ x:00004_core] o.a.s.u.p.LogUpdateProcessorFactory [00004_core] webapp=/solr path=/update params={wt=javabin&version=2}{add=[id7]} 0 0
[junit4] 2> 2008623 INFO (qtp2131987516-15809) [ x:00000_core] o.a.s.u.p.LogUpdateProcessorFactory [00000_core] webapp=/solr path=/update params={wt=javabin&version=2}{add=[id4]} 0 133
[junit4] 2> 2008623 INFO (qtp2131987516-15806) [ x:00000_core] o.a.s.u.p.LogUpdateProcessorFactory [00000_core] webapp=/solr path=/update params={wt=javabin&version=2}{add=[id5]} 0 133
[junit4] 2> 2008623 INFO (qtp2131987516-15808) [ x:00000_core] o.a.s.u.p.LogUpdateProcessorFactory [00000_core] webapp=/solr path=/update params={wt=javabin&version=2}{add=[id1]} 0 134
[junit4] 2> 2008627 WARN (qtp2131987516-15804) [ x:00003_core] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 2008627 INFO (qtp2131987516-15805) [ x:00000_core] o.a.s.u.p.LogUpdateProcessorFactory [00000_core] webapp=/solr path=/update params={wt=javabin&version=2}{add=[id8]} 0 5
[junit4] 2> 2008627 INFO (qtp2131987516-15804) [ x:00003_core] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2008628 INFO (qtp2131987516-15806) [ x:00004_core] o.a.s.u.p.LogUpdateProcessorFactory [00004_core] webapp=/solr path=/update params={wt=javabin&version=2}{add=[id12]} 0 0
[junit4] 2> 2008629 WARN (qtp2131987516-15840) [ x:00001_core] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 2008630 INFO (qtp2131987516-15840) [ x:00001_core] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2008633 INFO (qtp2131987516-15808) [ x:00004_core] o.a.s.c.S.Request [00004_core] webapp=/solr path=/select params={q=*:*&qt=/select&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 2008634 INFO (qtp2131987516-15804) [ x:00003_core] o.a.s.s.IndexSchema [00003_core] Schema name=tiny
[junit4] 2> 2008636 INFO (qtp2131987516-15840) [ x:00001_core] o.a.s.s.IndexSchema [00001_core] Schema name=tiny
[junit4] 2> 2008639 INFO (qtp2131987516-15840) [ x:00001_core] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 2008639 INFO (qtp2131987516-15804) [ x:00003_core] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 2008643 INFO (qtp2131987516-15804) [ x:00003_core] o.a.s.c.CoreContainer Creating SolrCore '00003_core' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_3B776154A0F4835F-001/tempDir-001/00003_core
[junit4] 2> 2008643 INFO (qtp2131987516-15840) [ x:00001_core] o.a.s.c.CoreContainer Creating SolrCore '00001_core' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_3B776154A0F4835F-001/tempDir-001/00001_core
[junit4] 2> 2008644 INFO (qtp2131987516-15804) [ x:00003_core] o.a.s.c.SolrCore [[00003_core] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_3B776154A0F4835F-001/tempDir-001/00003_core], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_3B776154A0F4835F-001/tempDir-001/00003_core/data/]
[junit4] 2> 2008644 INFO (qtp2131987516-15840) [ x:00001_core] o.a.s.c.SolrCore [[00001_core] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_3B776154A0F4835F-001/tempDir-001/00001_core], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_3B776154A0F4835F-001/tempDir-001/00001_core/data/]
[junit4] 2> 2008645 INFO (qtp2131987516-15804) [ x:00003_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=39, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.478960634184331]
[junit4] 2> 2008645 INFO (qtp2131987516-15840) [ x:00001_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=39, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.478960634184331]
[junit4] 2> 2008697 INFO (qtp2131987516-15840) [ x:00001_core] o.a.s.c.SolrCore [00001_core] CLOSING SolrCore org.apache.solr.core.SolrCore@5782d581
[junit4] 2> 2008697 INFO (qtp2131987516-15804) [ x:00003_core] o.a.s.c.SolrCore [00003_core] CLOSING SolrCore org.apache.solr.core.SolrCore@541b66e3
[junit4] 2> 2008697 INFO (qtp2131987516-15840) [ x:00001_core] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.00001_core
[junit4] 2> 2008697 INFO (qtp2131987516-15804) [ x:00003_core] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.00003_core
[junit4] 2> 2008699 WARN (qtp2131987516-15840) [ x:00001_core] o.e.j.s.ServletHandler Error for /solr/00001_core/update
[junit4] 2> java.lang.NoClassDefFoundError: org/apache/solr/servlet/HttpSolrCall$1
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.sendError(HttpSolrCall.java:687)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:543)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:298)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> Caused by: java.lang.ClassNotFoundException: org.apache.solr.servlet.HttpSolrCall$1
[junit4] 2> at java.net.URLClassLoader$1.run(URLClassLoader.java:370)
[junit4] 2> at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
[junit4] 2> at java.security.AccessController.doPrivileged(Native Method)
[junit4] 2> at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
[junit4] 2> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
[junit4] 2> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
[junit4] 2> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
[junit4] 2> ... 28 more
[junit4] 2> Caused by: java.io.FileNotFoundException: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/classes/java/org/apache/solr/servlet/HttpSolrCall$1.class (Too many open files)
[junit4] 2> at java.io.FileInputStream.open0(Native Method)
[junit4] 2> at java.io.FileInputStream.open(FileInputStream.java:195)
[junit4] 2> at java.io.FileInputStream.<init>(FileInputStream.java:138)
[junit4] 2> at sun.misc.URLClassPath$FileLoader$1.getInputStream(URLClassPath.java:1288)
[junit4] 2> at sun.misc.Resource.cachedInputStream(Resource.java:77)
[junit4] 2> at sun.misc.Resource.getByteBuffer(Resource.java:160)
[junit4] 2> at java.net.URLClassLoader.defineClass(URLClassLoader.java:454)
[junit4] 2> at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
[junit4] 2> at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
[junit4] 2> ... 34 more
[junit4] 2> 2008701 WARN (qtp2131987516-15804) [ x:00003_core] o.e.j.s.ServletHandler Error for /solr/00003_core/select
[junit4] 2> java.lang.NoClassDefFoundError: org/apache/solr/servlet/HttpSolrCall$1
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.sendError(HttpSolrCall.java:687)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:543)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:298)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:462)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 2008701 INFO (Thread-2925) [ ] o.a.s.c.OneIndexer Indexing thread 15826 swallowed one exception Error from server at http://127.0.0.1:58354/solr/00001_core: Expected mime type application/octet-stream but got text/html. <html>
[junit4] 2> <head>
[junit4] 2> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
[junit4] 2> <title>Error 500 </title>
[junit4] 2> </head>
[junit4] 2> <body>
[junit4] 2> <h2>HTTP ERROR: 500</h2>
[junit4] 2> <p>Problem accessing /solr/00001_core/update. Reason:
[junit4] 2> <pre> java.lang.NoClassDefFoundError: org/apache/solr/servlet/HttpSolrCall$1</pre></p>
[junit4] 2> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/>
[junit4] 2> </body>
[junit4] 2> </html>
[junit4] 2>
[junit4] 2> 2008702 INFO (Thread-2929) [ ] o.a.s.c.OneQuery Querying thread: 15831 swallowed exception: Error from server at http://127.0.0.1:58354/solr/00003_core: Expected mime type application/octet-stream but got text/html. <html>
[junit4] 2> <head>
[junit4] 2> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
[junit4] 2> <title>Error 500 </title>
[junit4] 2> </head>
[junit4] 2> <body>
[junit4] 2> <h2>HTTP ERROR: 500</h2>
[junit4] 2> <p>Problem accessing /solr/00003_core/select. Reason:
[junit4] 2> <pre> java.lang.NoClassDefFoundError: org/apache/solr/servlet/HttpSolrCall$1</pre></p>
[junit4] 2> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/>
[junit4] 2> </body>
[junit4] 2> </html>
[junit4] 2>
[junit4] 2> 2008702 WARN (qtp2131987516-15806) [ ] o.e.j.s.ServletHandler Error for /solr/00001_core/update
[junit4] 2> java.lang.NoClassDefFoundError: org/apache/solr/servlet/HttpSolrCall$1
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.sendError(HttpSolrCall.java:687)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:543)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:298)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 2008703 WARN (qtp2131987516-15805) [ ] o.e.j.s.ServletHandler Error for /solr/00003_core/select
[junit4] 2> java.lang.NoClassDefFoundError: org/apache/solr/servlet/HttpSolrCall$1
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.sendError(HttpSolrCall.java:687)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:543)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:298)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:462)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 2008703 INFO (Thread-2924) [ ] o.a.s.c.OneIndexer Indexing thread 15825 swallowed one exception Error from server at http://127.0.0.1:58354/solr/00001_core: Expected mime type application/octet-stream but got text/html. <html>
[junit4] 2> <head>
[junit4] 2> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
[junit4] 2> <title>Error 500 </title>
[junit4] 2> </head>
[junit4] 2> <body>
[junit4] 2> <h2>HTTP ERROR: 500</h2>
[junit4] 2> <p>Problem accessing /solr/00001_core/update. Reason:
[junit4] 2> <pre> java.lang.NoClassDefFoundError: org/apache/solr/servlet/HttpSolrCall$1</pre></p>
[junit4] 2> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/>
[junit4] 2> </body>
[junit4] 2> </html>
[junit4] 2>
[junit4] 2> 2008704 INFO (Thread-2933) [ ] o.a.s.c.OneQuery Querying thread: 15835 swallowed exception: Error from server at http://127.0.0.1:58354/solr/00003_core: Expected mime type application/octet-stream but got text/html. <html>
[junit4] 2> <head>
[junit4] 2> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
[junit4] 2> <title>Error 500 </title>
[junit4] 2> </head>
[junit4] 2> <body>
[junit4] 2> <h2>HTTP ERROR: 500</h2>
[junit4] 2> <p>Problem accessing /solr/00003_core/select. Reason:
[junit4] 2> <pre> java.lang.NoClassDefFoundError: org/apache/solr/servlet/HttpSolrCall$1</pre></p>
[junit4] 2> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/>
[junit4] 2> </body>
[junit4] 2> </html>
[junit4] 2>
[junit4] 2> 2008724 INFO (qtp2131987516-15808) [ x:00004_core] o.a.s.c.S.Request [00004_core] webapp=/solr path=/select params={q=*:*&qt=/select&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 2008725 WARN (qtp2131987516-15843) [ x:00003_core] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 2008725 WARN (qtp2131987516-15807) [ x:00001_core] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 2008725 INFO (qtp2131987516-15843) [ x:00003_core] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2008725 INFO (qtp2131987516-15807) [ x:00001_core] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2008731 INFO (qtp2131987516-15807) [ x:00001_core] o.a.s.s.IndexSchema [00001_core] Schema name=tiny
[junit4] 2> 2008731 INFO (qtp2131987516-15843) [ x:00003_core] o.a.s.s.IndexSchema [00003_core] Schema name=tiny
[junit4] 2> 2008735 INFO (qtp2131987516-15807) [ x:00001_core] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 2008736 INFO (qtp2131987516-15843) [ x:00003_core] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 2008740 INFO (qtp2131987516-15807) [ x:00001_core] o.a.s.c.CoreContainer Creating SolrCore '00001_core' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_3B776154A0F4835F-001/tempDir-001/00001_core
[junit4] 2> 2008740 INFO (qtp2131987516-15843) [ x:00003_core] o.a.s.c.CoreContainer Creating SolrCore '00003_core' using configuration from instancedir /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_3B776154A0F4835F-001/tempDir-001/00003_core
[junit4] 2> 2008740 INFO (qtp2131987516-15807) [ x:00001_core] o.a.s.c.SolrCore [[00001_core] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_3B776154A0F4835F-001/tempDir-001/00001_core], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_3B776154A0F4835F-001/tempDir-001/00001_core/data/]
[junit4] 2> 2008740 INFO (qtp2131987516-15843) [ x:00003_core] o.a.s.c.SolrCore [[00003_core] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_3B776154A0F4835F-001/tempDir-001/00003_core], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_3B776154A0F4835F-001/tempDir-001/00003_core/data/]
[junit4] 2> 2008742 INFO (qtp2131987516-15807) [ x:00001_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=15, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 2008742 INFO (qtp2131987516-15843) [ x:00003_core] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=39, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.478960634184331]
[junit4] 2> 2008760 INFO (qtp2131987516-15807) [ x:00001_core] o.a.s.c.SolrCore [00001_core] CLOSING SolrCore org.apache.solr.core.SolrCore@750623ab
[junit4] 2> 2008760 INFO (qtp2131987516-15843) [ x:00003_core] o.a.s.c.SolrCore [00003_core] CLOSING SolrCore org.apache.solr.core.SolrCore@54f07d47
[junit4] 2> 2008760 INFO (qtp2131987516-15807) [ x:00001_core] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.00001_core
[junit4] 2> 2008760 INFO (qtp2131987516-15843) [ x:00003_core] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.00003_core
[junit4] 2> 2008760 WARN (qtp2131987516-15807) [ x:00001_core] o.e.j.s.ServletHandler Error for /solr/00001_core/update
[junit4] 2> java.lang.NoClassDefFoundError: org/apache/solr/servlet/HttpSolrCall$1
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.sendError(HttpSolrCall.java:687)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:543)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:298)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 2008761 WARN (qtp2131987516-15808) [ ] o.e.j.s.ServletHandler Error for /solr/00003_core/select
[junit4] 2> java.lang.NoClassDefFoundError: org/apache/solr/servlet/HttpSolrCall$1
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.sendError(HttpSolrCall.java:687)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:543)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:298)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:462)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 2008761 WARN (qtp2131987516-15843) [ x:00003_core] o.e.j.s.ServletHandler Error for /solr/00003_core/update
[junit4] 2> java.lang.NoClassDefFoundError: org/apache/solr/servlet/HttpSolrCall$1
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.sendError(HttpSolrCall.java:687)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:543)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:298)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 2008762 INFO (Thread-2928) [ ] o.a.s.c.OneIndexer Indexing thread 15829 swallowed one exception Error from server at http://127.0.0.1:58354/solr/00001_core: Expected mime type application/octet-stream but got text/html. <html>
[junit4] 2> <head>
[junit4] 2> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
[junit4] 2> <title>Error 500 </title>
[junit4] 2> </head>
[junit4] 2> <body>
[junit4] 2> <h2>HTTP ERROR: 500</h2>
[junit4] 2> <p>Problem accessing /solr/00001_core/update. Reason:
[junit4] 2> <pre> java.lang.NoClassDefFoundError: org/apache/solr/servlet/HttpSolrCall$1</pre></p>
[junit4] 2> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/>
[junit4] 2> </body>
[junit4] 2> </html>
[junit4] 2>
[junit4] 2> 2008763 INFO (Thread-2927) [ ] o.a.s.c.OneIndexer Indexing thread 15828 swallowed one exception Error from server at http://127.0.0.1:58354/solr/00003_core: Expected mime type application/octet-stream but got text/html. <html>
[junit4] 2> <head>
[junit4] 2> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
[junit4] 2> <title>Error 500 </title>
[junit4] 2> </head>
[junit4] 2> <body>
[junit4] 2> <h2>HTTP ERROR: 500</h2>
[junit4] 2> <p>Problem accessing /solr/00003_core/update. Reason:
[junit4] 2> <pre> java.lang.NoClassDefFoundError: org/apache/solr/servlet/HttpSolrCall$1</pre></p>
[junit4] 2> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/>
[junit4] 2> </body>
[junit4] 2> </html>
[junit4] 2>
[junit4] 2> 2008763 INFO (Thread-2932) [ ] o.a.s.c.OneQuery Querying thread: 15834 swallowed exception: Error from server at http://127.0.0.1:58354/solr/00003_core: Expected mime type application/octet-stream but got text/html. <html>
[junit4] 2> <head>
[junit4] 2> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
[junit4] 2> <title>Error 500 </title>
[junit4] 2> </head>
[junit4] 2> <body>
[junit4] 2> <h2>HTTP ERROR: 500</h2>
[junit4] 2> <p>Problem accessing /solr/00003_core/select. Reason:
[junit4] 2> <pre> java.lang.NoClassDefFoundError: org/apache/solr/servlet/HttpSolrCall$1</pre></p>
[junit4] 2> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/>
[junit4] 2> </body>
[junit4] 2> </html>
[junit4] 2>
[junit4] 2> 2008762 WARN (qtp2131987516-15802) [ ] o.e.j.s.ServletHandler Error for /solr/00001_core/update
[junit4] 2> java.lang.NoClassDefFoundError: org/apache/solr/servlet/HttpSolrCall$1
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.sendError(HttpSolrCall.java:687)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:543)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:298)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.
[...truncated too long message...]
ING, group=TGRP-RecoveryZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
[junit4] > at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 2) Thread[id=15925, name=qtp1977136872-15925, state=TIMED_WAITING, group=TGRP-RecoveryZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
[junit4] > at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 3) Thread[id=15915, name=qtp1977136872-15915, state=TIMED_WAITING, group=TGRP-RecoveryZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
[junit4] > at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 4) Thread[id=15926, name=qtp1977136872-15926, state=TIMED_WAITING, group=TGRP-RecoveryZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
[junit4] > at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 5) Thread[id=15928, name=org.eclipse.jetty.server.session.HashSessionManager@a25907Timer, state=TIMED_WAITING, group=TGRP-RecoveryZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
[junit4] > at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
[junit4] > at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 6) Thread[id=15927, name=qtp1977136872-15927, state=TIMED_WAITING, group=TGRP-RecoveryZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
[junit4] > at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 7) Thread[id=15918, name=qtp1977136872-15918, state=TIMED_WAITING, group=TGRP-RecoveryZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
[junit4] > at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 8) Thread[id=15923, name=qtp1977136872-15923, state=TIMED_WAITING, group=TGRP-RecoveryZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
[junit4] > at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 9) Thread[id=15912, name=qtp1977136872-15912, state=TIMED_WAITING, group=TGRP-RecoveryZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
[junit4] > at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > at __randomizedtesting.SeedInfo.seed([3B776154A0F4835F]:0)Throwable #3: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=15928, name=org.eclipse.jetty.server.session.HashSessionManager@a25907Timer, state=TIMED_WAITING, group=TGRP-RecoveryZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
[junit4] > at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
[junit4] > at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > at __randomizedtesting.SeedInfo.seed([3B776154A0F4835F]:0)
[junit4] Completed [352/692 (3!)] on J1 in 27.47s, 0 tests, 3 errors <<< FAILURES!
[...truncated 63567 lines...]
[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3812
- Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3812/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC
4 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.core.TestLazyCores
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores: 1) Thread[id=15420, name=searcherExecutor-6277-thread-1, state=WAITING, group=TGRP-TestLazyCores] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores:
1) Thread[id=15420, name=searcherExecutor-6277-thread-1, state=WAITING, group=TGRP-TestLazyCores]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([98E153C0F551FBC3]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.core.TestLazyCores
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=15420, name=searcherExecutor-6277-thread-1, state=WAITING, group=TGRP-TestLazyCores] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=15420, name=searcherExecutor-6277-thread-1, state=WAITING, group=TGRP-TestLazyCores]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([98E153C0F551FBC3]:0)
FAILED: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test
Error Message:
Could not find collection:collection2
Stack Trace:
java.lang.AssertionError: Could not find collection:collection2
at __randomizedtesting.SeedInfo.seed([98E153C0F551FBC3:10B56C1A5BAD963B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:159)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:144)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:139)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:856)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexingBatchPerRequestWithHttpSolrClient(FullSolrCloudDistribCmdsTest.java:620)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test(FullSolrCloudDistribCmdsTest.java:152)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.core.TestLazyCores.testNoCommit
Error Message:
Exception during query
Stack Trace:
java.lang.RuntimeException: Exception during query
at __randomizedtesting.SeedInfo.seed([98E153C0F551FBC3:4781F2113E769866]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:857)
at org.apache.solr.core.TestLazyCores.check10(TestLazyCores.java:794)
at org.apache.solr.core.TestLazyCores.testNoCommit(TestLazyCores.java:776)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound='10']
xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">0</int>
<lst name="params">
<str name="q">*:*</str>
</lst>
</lst>
<result name="response" numFound="0" start="0">
</result>
</response>
request was:q=*:*
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:850)
... 41 more
Build Log:
[...truncated 11773 lines...]
[junit4] Suite: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
[junit4] 2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/init-core-data-001
[junit4] 2> 1726136 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[98E153C0F551FBC3]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 1726140 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[98E153C0F551FBC3]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776) w/ MAC_OS_X supressed clientAuth
[junit4] 2> 1726140 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[98E153C0F551FBC3]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /xt_nqa/ab
[junit4] 2> 1726142 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1726142 INFO (Thread-2079) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1726142 INFO (Thread-2079) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1726252 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.ZkTestServer start zk server on port:55853
[junit4] 2> 1726280 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1726285 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 1726287 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1726290 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1726293 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1726297 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1726300 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1726303 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1726306 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1726309 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1726312 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1726607 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/control-001/cores/collection1
[junit4] 2> 1726609 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1726611 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1fd4c81d{/xt_nqa/ab,null,AVAILABLE}
[junit4] 2> 1726613 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@41cb1f2{HTTP/1.1,[http/1.1]}{127.0.0.1:55856}
[junit4] 2> 1726613 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.e.j.s.Server Started @1732964ms
[junit4] 2> 1726613 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/tempDir-001/control/data, hostContext=/xt_nqa/ab, hostPort=55856, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/control-001/cores}
[junit4] 2> 1726614 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1726614 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 1726614 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1726614 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1726614 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-02T01:39:45.813Z
[junit4] 2> 1726618 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1726618 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/control-001/solr.xml
[junit4] 2> 1726636 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55853/solr
[junit4] 2> 1726683 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [n:127.0.0.1:55856_xt_nqa%2Fab ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:55856_xt_nqa%2Fab
[junit4] 2> 1726685 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [n:127.0.0.1:55856_xt_nqa%2Fab ] o.a.s.c.Overseer Overseer (id=97386468825169924-127.0.0.1:55856_xt_nqa%2Fab-n_0000000000) starting
[junit4] 2> 1726700 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [n:127.0.0.1:55856_xt_nqa%2Fab ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55856_xt_nqa%2Fab
[junit4] 2> 1726703 INFO (OverseerStateUpdate-97386468825169924-127.0.0.1:55856_xt_nqa%2Fab-n_0000000000) [n:127.0.0.1:55856_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1726807 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [n:127.0.0.1:55856_xt_nqa%2Fab ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/control-001/cores
[junit4] 2> 1726807 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [n:127.0.0.1:55856_xt_nqa%2Fab ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1726812 INFO (OverseerStateUpdate-97386468825169924-127.0.0.1:55856_xt_nqa%2Fab-n_0000000000) [n:127.0.0.1:55856_xt_nqa%2Fab ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 1727844 WARN (coreLoadExecutor-4519-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 1727846 INFO (coreLoadExecutor-4519-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1727860 INFO (coreLoadExecutor-4519-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1727936 WARN (coreLoadExecutor-4519-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1727937 INFO (coreLoadExecutor-4519-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1727967 INFO (coreLoadExecutor-4519-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 1727968 INFO (coreLoadExecutor-4519-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/control-001/cores/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/control-001/cores/collection1/data/]
[junit4] 2> 1727968 INFO (coreLoadExecutor-4519-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1954a154
[junit4] 2> 1727969 INFO (coreLoadExecutor-4519-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=43.875, floorSegmentMB=1.6064453125, forceMergeDeletesPctAllowed=26.871075218888336, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7780773115776406
[junit4] 2> 1727974 WARN (coreLoadExecutor-4519-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1728031 INFO (coreLoadExecutor-4519-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1728031 INFO (coreLoadExecutor-4519-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1728032 INFO (coreLoadExecutor-4519-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1728032 INFO (coreLoadExecutor-4519-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1728032 INFO (coreLoadExecutor-4519-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=14, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=11.775390625, floorSegmentMB=2.1416015625, forceMergeDeletesPctAllowed=4.099005163446259, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.37568336182126705
[junit4] 2> 1728033 INFO (coreLoadExecutor-4519-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@44666fc[collection1] main]
[junit4] 2> 1728035 INFO (coreLoadExecutor-4519-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1728035 INFO (coreLoadExecutor-4519-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1728035 INFO (coreLoadExecutor-4519-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1728037 INFO (searcherExecutor-4520-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab x:collection1 c:control_collection) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@44666fc[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1728037 INFO (coreLoadExecutor-4519-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558183503185575936
[junit4] 2> 1728050 INFO (coreZkRegister-4512-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab x:collection1 c:control_collection) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1728050 INFO (coreZkRegister-4512-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab x:collection1 c:control_collection) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1728050 INFO (coreZkRegister-4512-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab x:collection1 c:control_collection) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:55856/xt_nqa/ab/collection1/
[junit4] 2> 1728050 INFO (coreZkRegister-4512-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab x:collection1 c:control_collection) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1728050 INFO (coreZkRegister-4512-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab x:collection1 c:control_collection) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:55856/xt_nqa/ab/collection1/ has no replicas
[junit4] 2> 1728057 INFO (coreZkRegister-4512-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab x:collection1 c:control_collection) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:55856/xt_nqa/ab/collection1/ shard1
[junit4] 2> 1728216 INFO (coreZkRegister-4512-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab x:collection1 c:control_collection) [n:127.0.0.1:55856_xt_nqa%2Fab c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1728263 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1728265 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:55853/solr ready
[junit4] 2> 1728266 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 1728266 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
[junit4] 2> 1728574 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/shard-1-001/cores/collection1
[junit4] 2> 1728575 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/shard-1-001
[junit4] 2> 1728576 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1728579 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3dcb2ca3{/xt_nqa/ab,null,AVAILABLE}
[junit4] 2> 1728579 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7febe316{HTTP/1.1,[http/1.1]}{127.0.0.1:55864}
[junit4] 2> 1728579 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.e.j.s.Server Started @1734929ms
[junit4] 2> 1728579 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/xt_nqa/ab, hostPort=55864, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/shard-1-001/cores}
[junit4] 2> 1728580 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1728580 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 1728580 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1728580 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1728580 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-02T01:39:47.779Z
[junit4] 2> 1728584 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1728584 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/shard-1-001/solr.xml
[junit4] 2> 1728658 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55853/solr
[junit4] 2> 1728676 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [n:127.0.0.1:55864_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1728685 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [n:127.0.0.1:55864_xt_nqa%2Fab ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55864_xt_nqa%2Fab
[junit4] 2> 1728688 INFO (zkCallback-1516-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1728688 INFO (zkCallback-1512-thread-2-processing-n:127.0.0.1:55856_xt_nqa%2Fab) [n:127.0.0.1:55856_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1728689 INFO (zkCallback-1522-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1728775 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [n:127.0.0.1:55864_xt_nqa%2Fab ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/shard-1-001/cores
[junit4] 2> 1728775 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [n:127.0.0.1:55864_xt_nqa%2Fab ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1728779 INFO (OverseerStateUpdate-97386468825169924-127.0.0.1:55856_xt_nqa%2Fab-n_0000000000) [n:127.0.0.1:55856_xt_nqa%2Fab ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 1728890 INFO (zkCallback-1522-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1729796 WARN (coreLoadExecutor-4530-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 1729797 INFO (coreLoadExecutor-4530-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1729810 INFO (coreLoadExecutor-4530-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1729878 WARN (coreLoadExecutor-4530-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1729879 INFO (coreLoadExecutor-4530-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1729906 INFO (coreLoadExecutor-4530-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1729906 INFO (coreLoadExecutor-4530-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/shard-1-001/cores/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/shard-1-001/cores/collection1/data/]
[junit4] 2> 1729906 INFO (coreLoadExecutor-4530-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1954a154
[junit4] 2> 1729912 INFO (coreLoadExecutor-4530-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=43.875, floorSegmentMB=1.6064453125, forceMergeDeletesPctAllowed=26.871075218888336, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7780773115776406
[junit4] 2> 1729921 WARN (coreLoadExecutor-4530-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1730025 INFO (coreLoadExecutor-4530-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1730025 INFO (coreLoadExecutor-4530-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1730026 INFO (coreLoadExecutor-4530-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1730026 INFO (coreLoadExecutor-4530-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1730028 INFO (coreLoadExecutor-4530-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=14, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=11.775390625, floorSegmentMB=2.1416015625, forceMergeDeletesPctAllowed=4.099005163446259, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.37568336182126705
[junit4] 2> 1730029 INFO (coreLoadExecutor-4530-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5c249719[collection1] main]
[junit4] 2> 1730031 INFO (coreLoadExecutor-4530-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1730032 INFO (coreLoadExecutor-4530-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1730038 INFO (coreLoadExecutor-4530-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1730039 INFO (searcherExecutor-4531-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab x:collection1 c:collection1) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5c249719[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1730039 INFO (coreLoadExecutor-4530-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558183505284825088
[junit4] 2> 1730051 INFO (coreZkRegister-4525-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab x:collection1 c:collection1) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1730051 INFO (coreZkRegister-4525-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab x:collection1 c:collection1) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1730051 INFO (coreZkRegister-4525-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab x:collection1 c:collection1) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:55864/xt_nqa/ab/collection1/
[junit4] 2> 1730051 INFO (coreZkRegister-4525-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab x:collection1 c:collection1) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1730052 INFO (coreZkRegister-4525-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab x:collection1 c:collection1) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:55864/xt_nqa/ab/collection1/ has no replicas
[junit4] 2> 1730059 INFO (coreZkRegister-4525-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab x:collection1 c:collection1) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:55864/xt_nqa/ab/collection1/ shard2
[junit4] 2> 1730166 INFO (zkCallback-1522-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1730218 INFO (coreZkRegister-4525-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab x:collection1 c:collection1) [n:127.0.0.1:55864_xt_nqa%2Fab c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1730325 INFO (zkCallback-1522-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1730628 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/shard-2-001/cores/collection1
[junit4] 2> 1730630 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/shard-2-001
[junit4] 2> 1730631 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1730641 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6e5ed39f{/xt_nqa/ab,null,AVAILABLE}
[junit4] 2> 1730641 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3eb8b4b6{HTTP/1.1,[http/1.1]}{127.0.0.1:55869}
[junit4] 2> 1730642 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.e.j.s.Server Started @1736992ms
[junit4] 2> 1730642 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/xt_nqa/ab, hostPort=55869, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/shard-2-001/cores}
[junit4] 2> 1730642 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1730642 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 1730642 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1730642 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1730643 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-02T01:39:49.842Z
[junit4] 2> 1730647 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1730647 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/shard-2-001/solr.xml
[junit4] 2> 1730658 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55853/solr
[junit4] 2> 1730680 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [n:127.0.0.1:55869_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1730692 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [n:127.0.0.1:55869_xt_nqa%2Fab ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55869_xt_nqa%2Fab
[junit4] 2> 1730695 INFO (zkCallback-1522-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1730695 INFO (zkCallback-1512-thread-2-processing-n:127.0.0.1:55856_xt_nqa%2Fab) [n:127.0.0.1:55856_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1730696 INFO (zkCallback-1516-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1730698 INFO (zkCallback-1528-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1730812 INFO (zkCallback-1522-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1730837 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [n:127.0.0.1:55869_xt_nqa%2Fab ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/shard-2-001/cores
[junit4] 2> 1730837 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [n:127.0.0.1:55869_xt_nqa%2Fab ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1730840 INFO (OverseerStateUpdate-97386468825169924-127.0.0.1:55856_xt_nqa%2Fab-n_0000000000) [n:127.0.0.1:55856_xt_nqa%2Fab ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard3
[junit4] 2> 1730951 INFO (zkCallback-1522-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1730951 INFO (zkCallback-1528-thread-2-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1731867 WARN (coreLoadExecutor-4541-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 1731868 INFO (coreLoadExecutor-4541-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1731880 INFO (coreLoadExecutor-4541-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1731951 WARN (coreLoadExecutor-4541-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1731953 INFO (coreLoadExecutor-4541-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1731987 INFO (coreLoadExecutor-4541-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1731988 INFO (coreLoadExecutor-4541-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/shard-2-001/cores/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/shard-2-001/cores/collection1/data/]
[junit4] 2> 1731988 INFO (coreLoadExecutor-4541-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1954a154
[junit4] 2> 1731989 INFO (coreLoadExecutor-4541-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=43.875, floorSegmentMB=1.6064453125, forceMergeDeletesPctAllowed=26.871075218888336, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7780773115776406
[junit4] 2> 1732000 WARN (coreLoadExecutor-4541-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1732108 INFO (coreLoadExecutor-4541-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1732108 INFO (coreLoadExecutor-4541-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1732109 INFO (coreLoadExecutor-4541-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1732109 INFO (coreLoadExecutor-4541-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1732110 INFO (coreLoadExecutor-4541-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=14, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=11.775390625, floorSegmentMB=2.1416015625, forceMergeDeletesPctAllowed=4.099005163446259, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.37568336182126705
[junit4] 2> 1732110 INFO (coreLoadExecutor-4541-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@15151659[collection1] main]
[junit4] 2> 1732112 INFO (coreLoadExecutor-4541-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1732113 INFO (coreLoadExecutor-4541-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1732113 INFO (coreLoadExecutor-4541-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1732114 INFO (searcherExecutor-4542-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab x:collection1 c:collection1) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@15151659[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1732114 INFO (coreLoadExecutor-4541-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558183507460620288
[junit4] 2> 1732128 INFO (coreZkRegister-4536-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab x:collection1 c:collection1) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1732128 INFO (coreZkRegister-4536-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab x:collection1 c:collection1) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1732128 INFO (coreZkRegister-4536-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab x:collection1 c:collection1) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:55869/xt_nqa/ab/collection1/
[junit4] 2> 1732129 INFO (coreZkRegister-4536-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab x:collection1 c:collection1) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1732129 INFO (coreZkRegister-4536-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab x:collection1 c:collection1) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:55869/xt_nqa/ab/collection1/ has no replicas
[junit4] 2> 1732136 INFO (coreZkRegister-4536-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab x:collection1 c:collection1) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:55869/xt_nqa/ab/collection1/ shard3
[junit4] 2> 1732247 INFO (zkCallback-1522-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1732247 INFO (zkCallback-1528-thread-2-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1732296 INFO (coreZkRegister-4536-thread-1-processing-n:127.0.0.1:55869_xt_nqa%2Fab x:collection1 c:collection1) [n:127.0.0.1:55869_xt_nqa%2Fab c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1732412 INFO (zkCallback-1528-thread-2-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1732412 INFO (zkCallback-1522-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1732535 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/shard-3-001/cores/collection1
[junit4] 2> 1732536 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/shard-3-001
[junit4] 2> 1732537 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1732539 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@410d6484{/xt_nqa/ab,null,AVAILABLE}
[junit4] 2> 1732539 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5dc6bf71{HTTP/1.1,[http/1.1]}{127.0.0.1:55880}
[junit4] 2> 1732539 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.e.j.s.Server Started @1738890ms
[junit4] 2> 1732539 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/tempDir-001/jetty3, solrconfig=solrconfig.xml, hostContext=/xt_nqa/ab, hostPort=55880, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/shard-3-001/cores}
[junit4] 2> 1732540 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1732540 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 1732541 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1732541 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1732541 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-02T01:39:51.740Z
[junit4] 2> 1732544 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1732545 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/shard-3-001/solr.xml
[junit4] 2> 1732565 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55853/solr
[junit4] 2> 1732584 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [n:127.0.0.1:55880_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1732595 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [n:127.0.0.1:55880_xt_nqa%2Fab ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55880_xt_nqa%2Fab
[junit4] 2> 1732599 INFO (zkCallback-1512-thread-3-processing-n:127.0.0.1:55856_xt_nqa%2Fab) [n:127.0.0.1:55856_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1732600 INFO (zkCallback-1522-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1732600 INFO (zkCallback-1528-thread-2-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1732601 INFO (zkCallback-1534-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab) [n:127.0.0.1:55880_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1732602 INFO (zkCallback-1516-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1732701 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [n:127.0.0.1:55880_xt_nqa%2Fab ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/shard-3-001/cores
[junit4] 2> 1732701 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [n:127.0.0.1:55880_xt_nqa%2Fab ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1732706 INFO (OverseerStateUpdate-97386468825169924-127.0.0.1:55856_xt_nqa%2Fab-n_0000000000) [n:127.0.0.1:55856_xt_nqa%2Fab ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 1732820 INFO (zkCallback-1534-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab) [n:127.0.0.1:55880_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1732820 INFO (zkCallback-1522-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1732820 INFO (zkCallback-1528-thread-2-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1733730 WARN (coreLoadExecutor-4552-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 1733731 INFO (coreLoadExecutor-4552-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1733742 INFO (coreLoadExecutor-4552-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1733802 WARN (coreLoadExecutor-4552-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1733803 INFO (coreLoadExecutor-4552-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1733833 INFO (coreLoadExecutor-4552-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1733833 INFO (coreLoadExecutor-4552-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/shard-3-001/cores/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/shard-3-001/cores/collection1/data/]
[junit4] 2> 1733834 INFO (coreLoadExecutor-4552-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1954a154
[junit4] 2> 1733834 INFO (coreLoadExecutor-4552-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=43.875, floorSegmentMB=1.6064453125, forceMergeDeletesPctAllowed=26.871075218888336, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7780773115776406
[junit4] 2> 1733920 WARN (coreLoadExecutor-4552-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1733987 INFO (coreLoadExecutor-4552-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1733987 INFO (coreLoadExecutor-4552-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1733989 INFO (coreLoadExecutor-4552-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1733989 INFO (coreLoadExecutor-4552-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1733990 INFO (coreLoadExecutor-4552-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=14, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=11.775390625, floorSegmentMB=2.1416015625, forceMergeDeletesPctAllowed=4.099005163446259, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.37568336182126705
[junit4] 2> 1733990 INFO (coreLoadExecutor-4552-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@41110b55[collection1] main]
[junit4] 2> 1733992 INFO (coreLoadExecutor-4552-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1733992 INFO (coreLoadExecutor-4552-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1733993 INFO (coreLoadExecutor-4552-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1733994 INFO (searcherExecutor-4553-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab x:collection1 c:collection1) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@41110b55[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1733994 INFO (coreLoadExecutor-4552-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558183509431943168
[junit4] 2> 1734005 INFO (coreZkRegister-4547-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab x:collection1 c:collection1) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1734005 INFO (coreZkRegister-4547-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab x:collection1 c:collection1) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1734005 INFO (coreZkRegister-4547-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab x:collection1 c:collection1) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:55880/xt_nqa/ab/collection1/
[junit4] 2> 1734005 INFO (coreZkRegister-4547-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab x:collection1 c:collection1) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1734005 INFO (coreZkRegister-4547-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab x:collection1 c:collection1) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:55880/xt_nqa/ab/collection1/ has no replicas
[junit4] 2> 1734014 INFO (coreZkRegister-4547-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab x:collection1 c:collection1) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:55880/xt_nqa/ab/collection1/ shard1
[junit4] 2> 1734128 INFO (zkCallback-1534-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab) [n:127.0.0.1:55880_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1734128 INFO (zkCallback-1528-thread-2-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1734128 INFO (zkCallback-1522-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1734187 INFO (coreZkRegister-4547-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab x:collection1 c:collection1) [n:127.0.0.1:55880_xt_nqa%2Fab c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1734304 INFO (zkCallback-1534-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab) [n:127.0.0.1:55880_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1734304 INFO (zkCallback-1528-thread-2-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1734304 INFO (zkCallback-1522-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1735006 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/shard-4-001/cores/collection1
[junit4] 2> 1735007 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 4 in directory /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/shard-4-001
[junit4] 2> 1735007 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1735010 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3ee19321{/xt_nqa/ab,null,AVAILABLE}
[junit4] 2> 1735010 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@48848934{HTTP/1.1,[http/1.1]}{127.0.0.1:55889}
[junit4] 2> 1735010 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.e.j.s.Server Started @1741361ms
[junit4] 2> 1735010 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/tempDir-001/jetty4, solrconfig=solrconfig.xml, hostContext=/xt_nqa/ab, hostPort=55889, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/shard-4-001/cores}
[junit4] 2> 1735011 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1735011 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 1735011 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1735011 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1735011 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-02T01:39:54.210Z
[junit4] 2> 1735016 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1735016 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/shard-4-001/solr.xml
[junit4] 2> 1735167 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55853/solr
[junit4] 2> 1735184 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [n:127.0.0.1:55889_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1735193 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [n:127.0.0.1:55889_xt_nqa%2Fab ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55889_xt_nqa%2Fab
[junit4] 2> 1735196 INFO (zkCallback-1534-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab) [n:127.0.0.1:55880_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1735196 INFO (zkCallback-1516-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1735196 INFO (zkCallback-1512-thread-1-processing-n:127.0.0.1:55856_xt_nqa%2Fab) [n:127.0.0.1:55856_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1735196 INFO (zkCallback-1522-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1735196 INFO (zkCallback-1528-thread-2-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1735197 INFO (zkCallback-1540-thread-1-processing-n:127.0.0.1:55889_xt_nqa%2Fab) [n:127.0.0.1:55889_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1735283 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [n:127.0.0.1:55889_xt_nqa%2Fab ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.FullSolrCloudDistribCmdsTest_98E153C0F551FBC3-001/shard-4-001/cores
[junit4] 2> 1735283 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[98E153C0F551FBC3]) [n:127.0.0.1:55889_xt_nqa%2Fab ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1735288 INFO (OverseerStateUpdate-97386468825169924-127.0.0.1:55856_xt_nqa%2Fab-n_0000000000) [n:127.0.0.1:55856_xt_nqa%2Fab ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 1735394 INFO (zkCallback-1528-thread-2-processing-n:127.0.0.1:55869_xt_nqa%2Fab) [n:127.0.0.1:55869_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1735394 INFO (zkCallback-1534-thread-1-processing-n:127.0.0.1:55880_xt_nqa%2Fab) [n:127.0.0.1:55880_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1735394 INFO (zkCallback-1522-thread-1-processing-n:127.0.0.1:55864_xt_nqa%2Fab) [n:127.0.0.1:55864_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1735394 INFO (zkCallback-1540-thread-1-processing-n:127.0.0.1:55889_xt_nqa%2Fab) [n:127.0.0.1:55889_xt_nqa%2Fab ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 1736306 WARN (coreLoadExecutor-4563-thread-1-processing-n:127.0.0.1:55889_xt_nqa%2Fab) [n:127.0.0.1:55889_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 1736307 INFO (coreLoadExecutor-4563-thread-1-processing-n:127.0.0.1:55889_xt_nqa%2Fab) [n:127.0.0.1:55889_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1736319 INFO (coreLoadExecutor-4563-thread-1-processing-n:127.0.0.1:55889_xt_nqa%2Fab) [n:127.0.0.1:55889_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1736381 WARN (coreLoadExecutor-4563-thread-1-processing-n:127.0.0.1:55889_xt_nqa%2Fab) [n:127.0.0.1:55889_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1736382 INFO (coreLoadExecutor-4563-thread-1-processing-n:127.0.0.1:55889_xt_nqa%2Fab) [n:127.0.0.1:55889_xt_nqa%2Fab c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid
[...truncated too long message...]
FO (SUITE-TestLazyCores-seed#[98E153C0F551FBC3]-worker) [ x:collection4] o.a.s.c.CoreContainer Shutting down CoreContainer instance=573532611
[junit4] 2> 1811794 INFO (SUITE-TestLazyCores-seed#[98E153C0F551FBC3]-worker) [ x:collection4] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J0/temp/solr.core.TestLazyCores_98E153C0F551FBC3-001
[junit4] 2> Feb 02, 2017 1:41:11 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> Feb 02, 2017 1:41:31 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores:
[junit4] 2> 1) Thread[id=15420, name=searcherExecutor-6277-thread-1, state=WAITING, group=TGRP-TestLazyCores]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> Feb 02, 2017 1:41:31 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=15420, name=searcherExecutor-6277-thread-1, state=WAITING, group=TGRP-TestLazyCores]
[junit4] 2> Feb 02, 2017 1:41:34 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=15420, name=searcherExecutor-6277-thread-1, state=WAITING, group=TGRP-TestLazyCores]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=false): {}, locale=ar-OM, timezone=Canada/Newfoundland
[junit4] 2> NOTE: Mac OS X 10.11.6 x86_64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=3,threads=2,free=118275504,total=461373440
[junit4] 2> NOTE: All tests run in this JVM: [MetricsHandlerTest, TestCryptoKeys, HdfsLockFactoryTest, TestNRTOpen, QueryEqualityTest, AnalyticsMergeStrategyTest, TestManagedStopFilterFactory, HdfsBasicDistributedZk2Test, DeleteNodeTest, TestWordDelimiterFilterFactory, TestCharFilters, HighlighterTest, DocValuesMultiTest, StressHdfsTest, TestTestInjection, TestManagedSchema, AnalysisErrorHandlingTest, DistributedQueryComponentCustomSortTest, DistributedSuggestComponentTest, TestSearcherReuse, TestStressLiveNodes, ChangedSchemaMergeTest, RequiredFieldsTest, TestEmbeddedSolrServerConstructors, PKIAuthenticationIntegrationTest, BasicAuthIntegrationTest, IndexSchemaRuntimeFieldTest, TestSubQueryTransformerDistrib, DirectUpdateHandlerTest, TestNamedUpdateProcessors, DistributedFacetExistsSmallTest, TestFieldCache, TestReplicaProperties, TestSQLHandlerNonCloud, TestGeoJSONResponseWriter, SolrCLIZkUtilsTest, ZkControllerTest, PluginInfoTest, TestSolrQueryParserResource, TestLFUCache, SoftAutoCommitTest, TestPostingsSolrHighlighter, BJQParserTest, DataDrivenBlockJoinTest, TestMergePolicyConfig, TestReplicationHandler, TestRTGBase, SimpleFacetsTest, SharedFSAutoReplicaFailoverUtilsTest, SuggesterFSTTest, TestBM25SimilarityFactory, LeaderElectionContextKeyTest, ClassificationUpdateProcessorFactoryTest, TestFunctionQuery, TestStressReorder, SuggestComponentTest, TestExtendedDismaxParser, TestConfigOverlay, LeaderElectionTest, TestQueryWrapperFilter, TestInfoStreamLogging, CurrencyFieldXmlFileTest, ConnectionReuseTest, TestExpandComponent, DOMUtilTest, TestSolrJ, TestConfigSetProperties, TestBulkSchemaAPI, DateFieldTest, ConfigSetsAPITest, TestInPlaceUpdatesDistrib, TestLeaderElectionWithEmptyReplica, ClusterStateUpdateTest, OverseerRolesTest, TestExclusionRuleCollectionAccess, OutputWriterTest, TestFreeTextSuggestions, TestStandardQParsers, TestClassicSimilarityFactory, TestSweetSpotSimilarityFactory, OverseerCollectionConfigSetProcessorTest, ReplaceNodeTest, JsonValidatorTest, FileUtilsTest, CollectionReloadTest, ParsingFieldUpdateProcessorsTest, SearchHandlerTest, MetricUtilsTest, TestUseDocValuesAsStored, PolyFieldTest, DocValuesMissingTest, TestImpersonationWithHadoopAuth, CdcrReplicationHandlerTest, BlockJoinFacetRandomTest, CreateCollectionCleanupTest, TestRandomDVFaceting, TestConfigsApi, TestOnReconnectListenerSupport, RecoveryAfterSoftCommitTest, TestRestoreCore, TestJavabinTupleStreamParser, TestBlendedInfixSuggestions, HLLUtilTest, TestBinaryField, TestCloudManagedSchema, DistributedFacetPivotLongTailTest, DocExpirationUpdateProcessorFactoryTest, DirectSolrSpellCheckerTest, SimplePostToolTest, MoreLikeThisHandlerTest, SchemaVersionSpecificBehaviorTest, TestCloudPivotFacet, DocValuesTest, AliasIntegrationTest, TestCursorMarkWithoutUniqueKey, TestRemoteStreaming, TestNoOpRegenerator, TestRTimerTree, BasicDistributedZkTest, XmlUpdateRequestHandlerTest, HdfsRecoverLeaseTest, TestElisionMultitermQuery, SOLR749Test, SSLMigrationTest, StatelessScriptUpdateProcessorFactoryTest, BitVectorTest, TestPartialUpdateDeduplication, TestLRUCache, HttpSolrCallGetCoreTest, TestRandomRequestDistribution, SolrXmlInZkTest, TestConfigReload, SegmentsInfoRequestHandlerTest, TestFieldSortValues, BinaryUpdateRequestHandlerTest, OverseerTest, TestSolrCloudSnapshots, SolrSlf4jReporterTest, TestTrieFacet, TestLeaderInitiatedRecoveryThread, TestFastLRUCache, SmileWriterTest, UnloadDistributedZkTest, NumericFieldsTest, PrimUtilsTest, CollectionTooManyReplicasTest, OutOfBoxZkACLAndCredentialsProvidersTest, RulesTest, TestStressCloudBlindAtomicUpdates, TestCollationField, TestSolrConfigHandlerConcurrent, SolrIndexMetricsTest, ChaosMonkeyNothingIsSafeTest, TestExceedMaxTermLength, TestFaceting, DistribJoinFromCollectionTest, TestSort, TestCloudRecovery, SyncSliceTest, CursorPagingTest, TestLazyCores]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestLazyCores -Dtests.seed=98E153C0F551FBC3 -Dtests.slow=true -Dtests.locale=ar-OM -Dtests.timezone=Canada/Newfoundland -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J0 | TestLazyCores (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores:
[junit4] > 1) Thread[id=15420, name=searcherExecutor-6277-thread-1, state=WAITING, group=TGRP-TestLazyCores]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > at __randomizedtesting.SeedInfo.seed([98E153C0F551FBC3]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=15420, name=searcherExecutor-6277-thread-1, state=WAITING, group=TGRP-TestLazyCores]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > at __randomizedtesting.SeedInfo.seed([98E153C0F551FBC3]:0)
[junit4] Completed [341/691 (2!)] on J0 in 35.30s, 9 tests, 3 errors <<< FAILURES!
[...truncated 63573 lines...]
[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3811
- Still unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3811/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC
7 tests failed.
FAILED: org.apache.solr.cloud.CustomCollectionTest.testCustomCollectionsAPI
Error Message:
Could not find collection : implicitcoll
Stack Trace:
org.apache.solr.common.SolrException: Could not find collection : implicitcoll
at __randomizedtesting.SeedInfo.seed([AEBD1B8D455DE6D3:C45C95E678C750AB]:0)
at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:194)
at org.apache.solr.cloud.SolrCloudTestCase.getCollectionState(SolrCloudTestCase.java:245)
at org.apache.solr.cloud.CustomCollectionTest.testCustomCollectionsAPI(CustomCollectionTest.java:68)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.MigrateRouteKeyTest
Error Message:
java.io.IOException: Couldn't instantiate org.apache.zookeeper.ClientCnxnSocketNIO
Stack Trace:
org.apache.solr.common.SolrException: java.io.IOException: Couldn't instantiate org.apache.zookeeper.ClientCnxnSocketNIO
at __randomizedtesting.SeedInfo.seed([AEBD1B8D455DE6D3]:0)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:170)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:116)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:111)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:98)
at org.apache.solr.cloud.MiniSolrCloudCluster.waitForAllNodes(MiniSolrCloudCluster.java:260)
at org.apache.solr.cloud.MiniSolrCloudCluster.<init>(MiniSolrCloudCluster.java:254)
at org.apache.solr.cloud.SolrCloudTestCase$Builder.configure(SolrCloudTestCase.java:188)
at org.apache.solr.cloud.MigrateRouteKeyTest.setupCluster(MigrateRouteKeyTest.java:50)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:847)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Couldn't instantiate org.apache.zookeeper.ClientCnxnSocketNIO
at org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:1782)
at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:447)
at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:380)
at org.apache.solr.common.cloud.SolrZooKeeper.<init>(SolrZooKeeper.java:43)
at org.apache.solr.common.cloud.ZkClientConnectionStrategy.createSolrZooKeeper(ZkClientConnectionStrategy.java:105)
at org.apache.solr.common.cloud.DefaultConnectionStrategy.connect(DefaultConnectionStrategy.java:37)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:145)
... 31 more
Caused by: java.io.IOException: Too many open files
at sun.nio.ch.IOUtil.makePipe(Native Method)
at sun.nio.ch.KQueueSelectorImpl.<init>(KQueueSelectorImpl.java:84)
at sun.nio.ch.KQueueSelectorProvider.openSelector(KQueueSelectorProvider.java:42)
at java.nio.channels.Selector.open(Selector.java:227)
at org.apache.zookeeper.ClientCnxnSocketNIO.<init>(ClientCnxnSocketNIO.java:43)
at sun.reflect.GeneratedConstructorAccessor28.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:1779)
... 37 more
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.MigrateRouteKeyTest
Error Message:
ObjectTracker found 6 object(s) that were not released!!! [InternalHttpClient, InternalHttpClient, InternalHttpClient, SolrZkClient, SolrZkClient, InternalHttpClient] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:268) at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:90) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:489) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:268) at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:195) at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:47) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:483) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:268) at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:90) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:489) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:184) at org.apache.solr.cloud.ZkController.<init>(ZkController.java:296) at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:112) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:498) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:184) at org.apache.solr.cloud.ZkController.<init>(ZkController.java:296) at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:112) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:498) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:268) at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:195) at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:47) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:483) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
java.lang.AssertionError: ObjectTracker found 6 object(s) that were not released!!! [InternalHttpClient, InternalHttpClient, InternalHttpClient, SolrZkClient, SolrZkClient, InternalHttpClient]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:268)
at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:90)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:489)
at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169)
at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376)
at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:268)
at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:195)
at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:47)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:483)
at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169)
at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376)
at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:268)
at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:90)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:489)
at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169)
at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376)
at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:184)
at org.apache.solr.cloud.ZkController.<init>(ZkController.java:296)
at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:112)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:498)
at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169)
at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376)
at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:184)
at org.apache.solr.cloud.ZkController.<init>(ZkController.java:296)
at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:112)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:498)
at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169)
at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376)
at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:268)
at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:195)
at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:47)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:483)
at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:237)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:169)
at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:376)
at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:239)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([AEBD1B8D455DE6D3]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:289)
at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.MigrateRouteKeyTest
Error Message:
Could not remove the following files (in the order of attempts): /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.MigrateRouteKeyTest_AEBD1B8D455DE6D3-001/tempDir-001: java.nio.file.FileSystemException: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.MigrateRouteKeyTest_AEBD1B8D455DE6D3-001/tempDir-001: Too many open files /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.MigrateRouteKeyTest_AEBD1B8D455DE6D3-001/init-core-data-001: java.nio.file.FileSystemException: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.MigrateRouteKeyTest_AEBD1B8D455DE6D3-001/init-core-data-001: Too many open files /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.MigrateRouteKeyTest_AEBD1B8D455DE6D3-001: java.nio.file.FileSystemException: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.MigrateRouteKeyTest_AEBD1B8D455DE6D3-001: Too many open files
Stack Trace:
java.io.IOException: Could not remove the following files (in the order of attempts):
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.MigrateRouteKeyTest_AEBD1B8D455DE6D3-001/tempDir-001: java.nio.file.FileSystemException: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.MigrateRouteKeyTest_AEBD1B8D455DE6D3-001/tempDir-001: Too many open files
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.MigrateRouteKeyTest_AEBD1B8D455DE6D3-001/init-core-data-001: java.nio.file.FileSystemException: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.MigrateRouteKeyTest_AEBD1B8D455DE6D3-001/init-core-data-001: Too many open files
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.MigrateRouteKeyTest_AEBD1B8D455DE6D3-001: java.nio.file.FileSystemException: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.MigrateRouteKeyTest_AEBD1B8D455DE6D3-001: Too many open files
at org.apache.lucene.util.IOUtils.rm(IOUtils.java:323)
at org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:216)
at com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.MigrateRouteKeyTest
Error Message:
37 threads leaked from SUITE scope at org.apache.solr.cloud.MigrateRouteKeyTest: 1) Thread[id=10122, name=qtp2045742734-10122, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 2) Thread[id=10131, name=qtp432021010-10131, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 3) Thread[id=10111, name=ProcessThread(sid:0 cport:-1):, state=WAITING, group=TGRP-MigrateRouteKeyTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:120) 4) Thread[id=10142, name=Connection evictor, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:745) 5) Thread[id=10120, name=qtp2045742734-10120, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 6) Thread[id=10108, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-MigrateRouteKeyTest] at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:178) at java.lang.Thread.run(Thread.java:745) 7) Thread[id=10115, name=jetty-launcher-1600-thread-1, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 8) Thread[id=10141, name=Connection evictor, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:745) 9) Thread[id=10116, name=jetty-launcher-1600-thread-2, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 10) Thread[id=10119, name=qtp2045742734-10119, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 11) Thread[id=10152, name=jetty-launcher-1600-thread-1-EventThread, state=WAITING, group=TGRP-MigrateRouteKeyTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494) 12) Thread[id=10159, name=OverseerHdfsCoreFailoverThread-97384042559111173-127.0.0.1:53736_solr-n_0000000000, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:139) at java.lang.Thread.run(Thread.java:745) 13) Thread[id=10124, name=qtp2045742734-10124, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 14) Thread[id=10125, name=qtp2045742734-10125, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 15) Thread[id=10158, name=OverseerCollectionConfigSetProcessor-97384042559111173-127.0.0.1:53736_solr-n_0000000000, state=TIMED_WAITING, group=Overseer collection creation process.] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.apache.solr.cloud.DistributedQueue.peekElements(DistributedQueue.java:343) at org.apache.solr.cloud.OverseerTaskQueue.peekTopN(OverseerTaskQueue.java:235) at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:227) at java.lang.Thread.run(Thread.java:745) 16) Thread[id=10118, name=qtp2045742734-10118-acceptor-0@61ad8635-ServerConnector@2dc7c790{HTTP/1.1,[http/1.1]}{127.0.0.1:53736}, state=RUNNABLE, group=TGRP-MigrateRouteKeyTest] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:373) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:745) 17) Thread[id=10121, name=qtp432021010-10121, state=RUNNABLE, group=TGRP-MigrateRouteKeyTest] at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:745) 18) Thread[id=10109, name=SessionTracker, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest] at java.lang.Object.wait(Native Method) at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146) 19) Thread[id=10133, name=qtp432021010-10133, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 20) Thread[id=10151, name=jetty-launcher-1600-thread-1-SendThread(127.0.0.1:53734), state=RUNNABLE, group=TGRP-MigrateRouteKeyTest] at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) 21) Thread[id=10143, name=Connection evictor, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:745) 22) Thread[id=10107, name=Thread-2176, state=WAITING, group=TGRP-MigrateRouteKeyTest] at java.lang.Object.wait(Native Method) at java.lang.Thread.join(Thread.java:1249) at java.lang.Thread.join(Thread.java:1323) at org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:302) at org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:309) at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:490) 23) Thread[id=10132, name=org.eclipse.jetty.server.session.HashSessionManager@7a8c6712Timer, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 24) Thread[id=10123, name=qtp2045742734-10123, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 25) Thread[id=10130, name=qtp432021010-10130, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 26) Thread[id=10134, name=org.eclipse.jetty.server.session.HashSessionManager@5a46e6ebTimer, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 27) Thread[id=10128, name=qtp432021010-10128, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 28) Thread[id=10154, name=jetty-launcher-1600-thread-2-EventThread, state=WAITING, group=TGRP-MigrateRouteKeyTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494) 29) Thread[id=10155, name=zkCallback-1610-thread-1, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 30) Thread[id=10157, name=OverseerStateUpdate-97384042559111173-127.0.0.1:53736_solr-n_0000000000, state=TIMED_WAITING, group=Overseer state updater.] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:164) at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:138) at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:179) at java.lang.Thread.run(Thread.java:745) 31) Thread[id=10117, name=qtp2045742734-10117, state=RUNNABLE, group=TGRP-MigrateRouteKeyTest] at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:745) 32) Thread[id=10129, name=qtp432021010-10129, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 33) Thread[id=10156, name=zkCallback-1611-thread-1, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 34) Thread[id=10126, name=qtp432021010-10126-acceptor-0@7ddc89be-ServerConnector@15941abe{HTTP/1.1,[http/1.1]}{127.0.0.1:53737}, state=RUNNABLE, group=TGRP-MigrateRouteKeyTest] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:373) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:745) 35) Thread[id=10144, name=Connection evictor, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:745) 36) Thread[id=10127, name=qtp432021010-10127, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:745) 37) Thread[id=10153, name=jetty-launcher-1600-thread-2-SendThread(127.0.0.1:53734), state=RUNNABLE, group=TGRP-MigrateRouteKeyTest] at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 37 threads leaked from SUITE scope at org.apache.solr.cloud.MigrateRouteKeyTest:
1) Thread[id=10122, name=qtp2045742734-10122, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
2) Thread[id=10131, name=qtp432021010-10131, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
3) Thread[id=10111, name=ProcessThread(sid:0 cport:-1):, state=WAITING, group=TGRP-MigrateRouteKeyTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:120)
4) Thread[id=10142, name=Connection evictor, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:745)
5) Thread[id=10120, name=qtp2045742734-10120, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
6) Thread[id=10108, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-MigrateRouteKeyTest]
at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:178)
at java.lang.Thread.run(Thread.java:745)
7) Thread[id=10115, name=jetty-launcher-1600-thread-1, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
8) Thread[id=10141, name=Connection evictor, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:745)
9) Thread[id=10116, name=jetty-launcher-1600-thread-2, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
10) Thread[id=10119, name=qtp2045742734-10119, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
11) Thread[id=10152, name=jetty-launcher-1600-thread-1-EventThread, state=WAITING, group=TGRP-MigrateRouteKeyTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
12) Thread[id=10159, name=OverseerHdfsCoreFailoverThread-97384042559111173-127.0.0.1:53736_solr-n_0000000000, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:139)
at java.lang.Thread.run(Thread.java:745)
13) Thread[id=10124, name=qtp2045742734-10124, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
14) Thread[id=10125, name=qtp2045742734-10125, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
15) Thread[id=10158, name=OverseerCollectionConfigSetProcessor-97384042559111173-127.0.0.1:53736_solr-n_0000000000, state=TIMED_WAITING, group=Overseer collection creation process.]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.apache.solr.cloud.DistributedQueue.peekElements(DistributedQueue.java:343)
at org.apache.solr.cloud.OverseerTaskQueue.peekTopN(OverseerTaskQueue.java:235)
at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:227)
at java.lang.Thread.run(Thread.java:745)
16) Thread[id=10118, name=qtp2045742734-10118-acceptor-0@61ad8635-ServerConnector@2dc7c790{HTTP/1.1,[http/1.1]}{127.0.0.1:53736}, state=RUNNABLE, group=TGRP-MigrateRouteKeyTest]
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:373)
at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:745)
17) Thread[id=10121, name=qtp432021010-10121, state=RUNNABLE, group=TGRP-MigrateRouteKeyTest]
at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:745)
18) Thread[id=10109, name=SessionTracker, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest]
at java.lang.Object.wait(Native Method)
at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)
19) Thread[id=10133, name=qtp432021010-10133, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
20) Thread[id=10151, name=jetty-launcher-1600-thread-1-SendThread(127.0.0.1:53734), state=RUNNABLE, group=TGRP-MigrateRouteKeyTest]
at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
21) Thread[id=10143, name=Connection evictor, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:745)
22) Thread[id=10107, name=Thread-2176, state=WAITING, group=TGRP-MigrateRouteKeyTest]
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1249)
at java.lang.Thread.join(Thread.java:1323)
at org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:302)
at org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:309)
at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:490)
23) Thread[id=10132, name=org.eclipse.jetty.server.session.HashSessionManager@7a8c6712Timer, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
24) Thread[id=10123, name=qtp2045742734-10123, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
25) Thread[id=10130, name=qtp432021010-10130, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
26) Thread[id=10134, name=org.eclipse.jetty.server.session.HashSessionManager@5a46e6ebTimer, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
27) Thread[id=10128, name=qtp432021010-10128, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:745)
28) Thread[id=10154, name=jetty-launcher-1600-thread-2-EventThread, state=WAITING, group=TGRP-MigrateRouteKeyTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
29) Thread[id=10155, name=zkCallback-1610-thread-1, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
30) Thread[id=10157, name=OverseerStateUpdate-97384042559111173-127.0.0.1:53736_solr-n_0000000000, state=TIMED_WAITING, group=Overseer state updater.]
at sun.misc.Uns
[...truncated too long message...]
ad-1, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 3) Thread[id=10117, name=qtp2045742734-10117, state=RUNNABLE, group=TGRP-MigrateRouteKeyTest]
[junit4] > at sun.nio.ch.KQueueArrayWrapper.interrupt(Native Method)
[junit4] > at sun.nio.ch.KQueueArrayWrapper.interrupt(KQueueArrayWrapper.java:203)
[junit4] > at sun.nio.ch.KQueueSelectorImpl.wakeup(KQueueSelectorImpl.java:257)
[junit4] > at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:213)
[junit4] > at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:219)
[junit4] > at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:116)
[junit4] > at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
[junit4] > at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
[junit4] > at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
[junit4] > at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 4) Thread[id=10156, name=zkCallback-1611-thread-1, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 5) Thread[id=10121, name=qtp432021010-10121, state=RUNNABLE, group=TGRP-MigrateRouteKeyTest]
[junit4] > at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
[junit4] > at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
[junit4] > at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117)
[junit4] > at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
[junit4] > at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
[junit4] > at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
[junit4] > at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 6) Thread[id=10115, name=jetty-launcher-1600-thread-1, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 7) Thread[id=10116, name=jetty-launcher-1600-thread-2, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 8) Thread[id=10153, name=jetty-launcher-1600-thread-2-SendThread(127.0.0.1:53734), state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] > 9) Thread[id=10151, name=jetty-launcher-1600-thread-1-SendThread(127.0.0.1:53734), state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] > 10) Thread[id=10134, name=org.eclipse.jetty.server.session.HashSessionManager@5a46e6ebTimer, state=TIMED_WAITING, group=TGRP-MigrateRouteKeyTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
[junit4] > at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
[junit4] > at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > at __randomizedtesting.SeedInfo.seed([AEBD1B8D455DE6D3]:0)
[junit4] Completed [295/691 (3!)] on J1 in 54.97s, 0 tests, 1 failure, 4 errors <<< FAILURES!
[...truncated 63648 lines...]
[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3810
- Failure!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3810/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC
1 tests failed.
FAILED: org.apache.solr.cloud.PeerSyncReplicationTest.test
Error Message:
timeout waiting to see all nodes active
Stack Trace:
java.lang.AssertionError: timeout waiting to see all nodes active
at __randomizedtesting.SeedInfo.seed([5BBC5CEDFDB3B313:D3E86337534FDEEB]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.PeerSyncReplicationTest.waitTillNodesActive(PeerSyncReplicationTest.java:326)
at org.apache.solr.cloud.PeerSyncReplicationTest.bringUpDeadNodeAndEnsureNoReplication(PeerSyncReplicationTest.java:277)
at org.apache.solr.cloud.PeerSyncReplicationTest.forceNodeFailureAndDoPeerSync(PeerSyncReplicationTest.java:259)
at org.apache.solr.cloud.PeerSyncReplicationTest.test(PeerSyncReplicationTest.java:138)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11059 lines...]
[junit4] Suite: org.apache.solr.cloud.PeerSyncReplicationTest
[junit4] 2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/init-core-data-001
[junit4] 2> 602426 INFO (SUITE-PeerSyncReplicationTest-seed#[5BBC5CEDFDB3B313]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields
[junit4] 2> 602431 INFO (SUITE-PeerSyncReplicationTest-seed#[5BBC5CEDFDB3B313]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN) w/ MAC_OS_X supressed clientAuth
[junit4] 2> 602431 INFO (SUITE-PeerSyncReplicationTest-seed#[5BBC5CEDFDB3B313]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_cn/ug
[junit4] 2> 602434 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 602434 INFO (Thread-908) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 602434 INFO (Thread-908) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 602542 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.ZkTestServer start zk server on port:60096
[junit4] 2> 602576 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 602581 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 602584 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 602586 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 602589 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 602592 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 602595 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 602598 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 602601 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 602604 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 602607 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 602938 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/control-001/cores/collection1
[junit4] 2> 602940 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 602942 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@cc96087{/_cn/ug,null,AVAILABLE}
[junit4] 2> 602942 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@74dc832e{HTTP/1.1,[http/1.1]}{127.0.0.1:60099}
[junit4] 2> 602942 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.e.j.s.Server Started @607732ms
[junit4] 2> 602942 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/tempDir-001/control/data, hostContext=/_cn/ug, hostPort=60099, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/control-001/cores}
[junit4] 2> 602943 ERROR (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 602943 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 602943 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 602943 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 602944 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-01T07:11:41.972Z
[junit4] 2> 602948 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 602948 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/control-001/solr.xml
[junit4] 2> 602976 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60096/solr
[junit4] 2> 603022 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [n:127.0.0.1:60099__cn%2Fug ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:60099__cn%2Fug
[junit4] 2> 603024 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [n:127.0.0.1:60099__cn%2Fug ] o.a.s.c.Overseer Overseer (id=97382111737610244-127.0.0.1:60099__cn%2Fug-n_0000000000) starting
[junit4] 2> 603041 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [n:127.0.0.1:60099__cn%2Fug ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60099__cn%2Fug
[junit4] 2> 603045 INFO (zkCallback-625-thread-1-processing-n:127.0.0.1:60099__cn%2Fug) [n:127.0.0.1:60099__cn%2Fug ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 603563 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [n:127.0.0.1:60099__cn%2Fug ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/control-001/cores
[junit4] 2> 603563 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [n:127.0.0.1:60099__cn%2Fug ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 603569 INFO (OverseerStateUpdate-97382111737610244-127.0.0.1:60099__cn%2Fug-n_0000000000) [n:127.0.0.1:60099__cn%2Fug ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 604593 INFO (coreLoadExecutor-1968-thread-1-processing-n:127.0.0.1:60099__cn%2Fug) [n:127.0.0.1:60099__cn%2Fug c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 604613 INFO (coreLoadExecutor-1968-thread-1-processing-n:127.0.0.1:60099__cn%2Fug) [n:127.0.0.1:60099__cn%2Fug c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 604708 WARN (coreLoadExecutor-1968-thread-1-processing-n:127.0.0.1:60099__cn%2Fug) [n:127.0.0.1:60099__cn%2Fug c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 604710 INFO (coreLoadExecutor-1968-thread-1-processing-n:127.0.0.1:60099__cn%2Fug) [n:127.0.0.1:60099__cn%2Fug c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 604733 INFO (coreLoadExecutor-1968-thread-1-processing-n:127.0.0.1:60099__cn%2Fug) [n:127.0.0.1:60099__cn%2Fug c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 604734 INFO (coreLoadExecutor-1968-thread-1-processing-n:127.0.0.1:60099__cn%2Fug) [n:127.0.0.1:60099__cn%2Fug c:control_collection x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/control-001/cores/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/control-001/cores/collection1/data/]
[junit4] 2> 604734 INFO (coreLoadExecutor-1968-thread-1-processing-n:127.0.0.1:60099__cn%2Fug) [n:127.0.0.1:60099__cn%2Fug c:control_collection x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7739f914
[junit4] 2> 604737 INFO (coreLoadExecutor-1968-thread-1-processing-n:127.0.0.1:60099__cn%2Fug) [n:127.0.0.1:60099__cn%2Fug c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=43, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=3.126953125, floorSegmentMB=0.302734375, forceMergeDeletesPctAllowed=20.08510998242944, segmentsPerTier=43.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.31395065872592465
[junit4] 2> 604754 WARN (coreLoadExecutor-1968-thread-1-processing-n:127.0.0.1:60099__cn%2Fug) [n:127.0.0.1:60099__cn%2Fug c:control_collection x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 604818 INFO (coreLoadExecutor-1968-thread-1-processing-n:127.0.0.1:60099__cn%2Fug) [n:127.0.0.1:60099__cn%2Fug c:control_collection x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 604818 INFO (coreLoadExecutor-1968-thread-1-processing-n:127.0.0.1:60099__cn%2Fug) [n:127.0.0.1:60099__cn%2Fug c:control_collection x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 604819 INFO (coreLoadExecutor-1968-thread-1-processing-n:127.0.0.1:60099__cn%2Fug) [n:127.0.0.1:60099__cn%2Fug c:control_collection x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 604820 INFO (coreLoadExecutor-1968-thread-1-processing-n:127.0.0.1:60099__cn%2Fug) [n:127.0.0.1:60099__cn%2Fug c:control_collection x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 604820 INFO (coreLoadExecutor-1968-thread-1-processing-n:127.0.0.1:60099__cn%2Fug) [n:127.0.0.1:60099__cn%2Fug c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@3c0e6238
[junit4] 2> 604829 INFO (coreLoadExecutor-1968-thread-1-processing-n:127.0.0.1:60099__cn%2Fug) [n:127.0.0.1:60099__cn%2Fug c:control_collection x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@700787dd[collection1] main]
[junit4] 2> 604831 INFO (coreLoadExecutor-1968-thread-1-processing-n:127.0.0.1:60099__cn%2Fug) [n:127.0.0.1:60099__cn%2Fug c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 604832 INFO (coreLoadExecutor-1968-thread-1-processing-n:127.0.0.1:60099__cn%2Fug) [n:127.0.0.1:60099__cn%2Fug c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 604832 INFO (coreLoadExecutor-1968-thread-1-processing-n:127.0.0.1:60099__cn%2Fug) [n:127.0.0.1:60099__cn%2Fug c:control_collection x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 604833 INFO (searcherExecutor-1969-thread-1-processing-n:127.0.0.1:60099__cn%2Fug x:collection1 c:control_collection) [n:127.0.0.1:60099__cn%2Fug c:control_collection x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@700787dd[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 604834 INFO (coreLoadExecutor-1968-thread-1-processing-n:127.0.0.1:60099__cn%2Fug) [n:127.0.0.1:60099__cn%2Fug c:control_collection x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558113790315200512
[junit4] 2> 604844 INFO (coreZkRegister-1961-thread-1-processing-n:127.0.0.1:60099__cn%2Fug x:collection1 c:control_collection) [n:127.0.0.1:60099__cn%2Fug c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 604845 INFO (coreZkRegister-1961-thread-1-processing-n:127.0.0.1:60099__cn%2Fug x:collection1 c:control_collection) [n:127.0.0.1:60099__cn%2Fug c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 604845 INFO (coreZkRegister-1961-thread-1-processing-n:127.0.0.1:60099__cn%2Fug x:collection1 c:control_collection) [n:127.0.0.1:60099__cn%2Fug c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:60099/_cn/ug/collection1/
[junit4] 2> 604845 INFO (coreZkRegister-1961-thread-1-processing-n:127.0.0.1:60099__cn%2Fug x:collection1 c:control_collection) [n:127.0.0.1:60099__cn%2Fug c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 604845 INFO (coreZkRegister-1961-thread-1-processing-n:127.0.0.1:60099__cn%2Fug x:collection1 c:control_collection) [n:127.0.0.1:60099__cn%2Fug c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:60099/_cn/ug/collection1/ has no replicas
[junit4] 2> 604855 INFO (coreZkRegister-1961-thread-1-processing-n:127.0.0.1:60099__cn%2Fug x:collection1 c:control_collection) [n:127.0.0.1:60099__cn%2Fug c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:60099/_cn/ug/collection1/ shard1
[junit4] 2> 605008 INFO (coreZkRegister-1961-thread-1-processing-n:127.0.0.1:60099__cn%2Fug x:collection1 c:control_collection) [n:127.0.0.1:60099__cn%2Fug c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 605318 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 605320 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:60096/solr ready
[junit4] 2> 605320 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 605320 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
[junit4] 2> 605643 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/shard-1-001/cores/collection1
[junit4] 2> 605645 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/shard-1-001
[junit4] 2> 605645 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 605647 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5990e5e6{/_cn/ug,null,AVAILABLE}
[junit4] 2> 605648 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3fb974f0{HTTP/1.1,[http/1.1]}{127.0.0.1:60105}
[junit4] 2> 605648 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.e.j.s.Server Started @610437ms
[junit4] 2> 605648 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/_cn/ug, hostPort=60105, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/shard-1-001/cores}
[junit4] 2> 605649 ERROR (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 605649 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 605649 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 605649 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 605649 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-01T07:11:44.677Z
[junit4] 2> 605655 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 605655 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/shard-1-001/solr.xml
[junit4] 2> 605675 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60096/solr
[junit4] 2> 605695 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 605704 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60105__cn%2Fug
[junit4] 2> 605709 INFO (zkCallback-625-thread-1-processing-n:127.0.0.1:60099__cn%2Fug) [n:127.0.0.1:60099__cn%2Fug ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 605710 INFO (zkCallback-635-thread-1-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 605711 INFO (zkCallback-629-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 605840 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/shard-1-001/cores
[junit4] 2> 605840 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 605843 INFO (OverseerStateUpdate-97382111737610244-127.0.0.1:60099__cn%2Fug-n_0000000000) [n:127.0.0.1:60099__cn%2Fug ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 605954 INFO (zkCallback-635-thread-2-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 606861 INFO (coreLoadExecutor-1979-thread-1-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 606883 INFO (coreLoadExecutor-1979-thread-1-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 606982 WARN (coreLoadExecutor-1979-thread-1-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 606984 INFO (coreLoadExecutor-1979-thread-1-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 607004 INFO (coreLoadExecutor-1979-thread-1-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 607005 INFO (coreLoadExecutor-1979-thread-1-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/shard-1-001/cores/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/shard-1-001/cores/collection1/data/]
[junit4] 2> 607005 INFO (coreLoadExecutor-1979-thread-1-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7739f914
[junit4] 2> 607008 INFO (coreLoadExecutor-1979-thread-1-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=43, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=3.126953125, floorSegmentMB=0.302734375, forceMergeDeletesPctAllowed=20.08510998242944, segmentsPerTier=43.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.31395065872592465
[junit4] 2> 607020 WARN (coreLoadExecutor-1979-thread-1-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 607091 INFO (coreLoadExecutor-1979-thread-1-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 607091 INFO (coreLoadExecutor-1979-thread-1-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 607093 INFO (coreLoadExecutor-1979-thread-1-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 607093 INFO (coreLoadExecutor-1979-thread-1-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 607094 INFO (coreLoadExecutor-1979-thread-1-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@6f163d3f
[junit4] 2> 607095 INFO (coreLoadExecutor-1979-thread-1-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@b0e5033[collection1] main]
[junit4] 2> 607097 INFO (coreLoadExecutor-1979-thread-1-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 607097 INFO (coreLoadExecutor-1979-thread-1-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 607098 INFO (coreLoadExecutor-1979-thread-1-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 607110 INFO (searcherExecutor-1980-thread-1-processing-n:127.0.0.1:60105__cn%2Fug x:collection1 c:collection1) [n:127.0.0.1:60105__cn%2Fug c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@b0e5033[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 607111 INFO (coreLoadExecutor-1979-thread-1-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558113792702808064
[junit4] 2> 607122 INFO (coreZkRegister-1974-thread-1-processing-n:127.0.0.1:60105__cn%2Fug x:collection1 c:collection1) [n:127.0.0.1:60105__cn%2Fug c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 607122 INFO (coreZkRegister-1974-thread-1-processing-n:127.0.0.1:60105__cn%2Fug x:collection1 c:collection1) [n:127.0.0.1:60105__cn%2Fug c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 607122 INFO (coreZkRegister-1974-thread-1-processing-n:127.0.0.1:60105__cn%2Fug x:collection1 c:collection1) [n:127.0.0.1:60105__cn%2Fug c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:60105/_cn/ug/collection1/
[junit4] 2> 607123 INFO (coreZkRegister-1974-thread-1-processing-n:127.0.0.1:60105__cn%2Fug x:collection1 c:collection1) [n:127.0.0.1:60105__cn%2Fug c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 607123 INFO (coreZkRegister-1974-thread-1-processing-n:127.0.0.1:60105__cn%2Fug x:collection1 c:collection1) [n:127.0.0.1:60105__cn%2Fug c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:60105/_cn/ug/collection1/ has no replicas
[junit4] 2> 607130 INFO (coreZkRegister-1974-thread-1-processing-n:127.0.0.1:60105__cn%2Fug x:collection1 c:collection1) [n:127.0.0.1:60105__cn%2Fug c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:60105/_cn/ug/collection1/ shard1
[junit4] 2> 607237 INFO (zkCallback-635-thread-2-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 607290 INFO (coreZkRegister-1974-thread-1-processing-n:127.0.0.1:60105__cn%2Fug x:collection1 c:collection1) [n:127.0.0.1:60105__cn%2Fug c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 607400 INFO (zkCallback-635-thread-2-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 607844 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/shard-2-001/cores/collection1
[junit4] 2> 607846 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/shard-2-001
[junit4] 2> 607846 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 607849 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4f4d28cf{/_cn/ug,null,AVAILABLE}
[junit4] 2> 607849 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@74f2374{HTTP/1.1,[http/1.1]}{127.0.0.1:60120}
[junit4] 2> 607849 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.e.j.s.Server Started @612639ms
[junit4] 2> 607849 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/_cn/ug, hostPort=60120, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/shard-2-001/cores}
[junit4] 2> 607850 ERROR (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 607850 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 607850 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 607851 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 607851 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-01T07:11:46.879Z
[junit4] 2> 607856 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 607856 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/shard-2-001/solr.xml
[junit4] 2> 607869 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60096/solr
[junit4] 2> 607888 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [n:127.0.0.1:60120__cn%2Fug ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 607898 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [n:127.0.0.1:60120__cn%2Fug ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60120__cn%2Fug
[junit4] 2> 607902 INFO (zkCallback-635-thread-2-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 607903 INFO (zkCallback-641-thread-1-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 607902 INFO (zkCallback-625-thread-2-processing-n:127.0.0.1:60099__cn%2Fug) [n:127.0.0.1:60099__cn%2Fug ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 607903 INFO (zkCallback-629-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 607992 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [n:127.0.0.1:60120__cn%2Fug ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/shard-2-001/cores
[junit4] 2> 607992 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [n:127.0.0.1:60120__cn%2Fug ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 607998 INFO (OverseerStateUpdate-97382111737610244-127.0.0.1:60099__cn%2Fug-n_0000000000) [n:127.0.0.1:60099__cn%2Fug ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 608108 INFO (zkCallback-635-thread-2-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 608108 INFO (zkCallback-641-thread-1-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 609028 INFO (coreLoadExecutor-1990-thread-1-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 609043 INFO (coreLoadExecutor-1990-thread-1-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 609136 WARN (coreLoadExecutor-1990-thread-1-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 609138 INFO (coreLoadExecutor-1990-thread-1-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 609159 INFO (coreLoadExecutor-1990-thread-1-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 609159 INFO (coreLoadExecutor-1990-thread-1-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/shard-2-001/cores/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/shard-2-001/cores/collection1/data/]
[junit4] 2> 609159 INFO (coreLoadExecutor-1990-thread-1-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7739f914
[junit4] 2> 609163 INFO (coreLoadExecutor-1990-thread-1-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=43, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=3.126953125, floorSegmentMB=0.302734375, forceMergeDeletesPctAllowed=20.08510998242944, segmentsPerTier=43.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.31395065872592465
[junit4] 2> 609175 WARN (coreLoadExecutor-1990-thread-1-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 609255 INFO (coreLoadExecutor-1990-thread-1-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 609255 INFO (coreLoadExecutor-1990-thread-1-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 609256 INFO (coreLoadExecutor-1990-thread-1-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 609256 INFO (coreLoadExecutor-1990-thread-1-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 609257 INFO (coreLoadExecutor-1990-thread-1-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@50182cb4
[junit4] 2> 609258 INFO (coreLoadExecutor-1990-thread-1-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3ef7b05a[collection1] main]
[junit4] 2> 609260 INFO (coreLoadExecutor-1990-thread-1-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 609261 INFO (coreLoadExecutor-1990-thread-1-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 609261 INFO (coreLoadExecutor-1990-thread-1-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 609262 INFO (searcherExecutor-1991-thread-1-processing-n:127.0.0.1:60120__cn%2Fug x:collection1 c:collection1) [n:127.0.0.1:60120__cn%2Fug c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3ef7b05a[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 609262 INFO (coreLoadExecutor-1990-thread-1-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558113794958295040
[junit4] 2> 609269 INFO (coreZkRegister-1985-thread-1-processing-n:127.0.0.1:60120__cn%2Fug x:collection1 c:collection1) [n:127.0.0.1:60120__cn%2Fug c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkController Core needs to recover:collection1
[junit4] 2> 609270 INFO (updateExecutor-638-thread-1-processing-n:127.0.0.1:60120__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:60120__cn%2Fug c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 609270 INFO (recoveryExecutor-639-thread-1-processing-n:127.0.0.1:60120__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:60120__cn%2Fug c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 609271 INFO (recoveryExecutor-639-thread-1-processing-n:127.0.0.1:60120__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:60120__cn%2Fug c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 609271 INFO (recoveryExecutor-639-thread-1-processing-n:127.0.0.1:60120__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:60120__cn%2Fug c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1]
[junit4] 2> 609271 INFO (recoveryExecutor-639-thread-1-processing-n:127.0.0.1:60120__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:60120__cn%2Fug c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 609271 INFO (recoveryExecutor-639-thread-1-processing-n:127.0.0.1:60120__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:60120__cn%2Fug c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core [collection1] as recovering, leader is [http://127.0.0.1:60105/_cn/ug/collection1/] and I am [http://127.0.0.1:60120/_cn/ug/collection1/]
[junit4] 2> 609277 INFO (recoveryExecutor-639-thread-1-processing-n:127.0.0.1:60120__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:60120__cn%2Fug c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:60105/_cn/ug]; [WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:60120__cn%252Fug&coreNodeName=core_node2&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 609281 INFO (qtp974522949-3685) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 609281 INFO (qtp974522949-3685) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 (shard1 of collection1) have state: recovering
[junit4] 2> 609281 INFO (qtp974522949-3685) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:60120__cn%2Fug, coreNodeName=core_node2, onlyIfActiveCheckResult=false, nodeProps: core_node2:{"core":"collection1","base_url":"http://127.0.0.1:60120/_cn/ug","node_name":"127.0.0.1:60120__cn%2Fug","state":"down"}
[junit4] 2> 609381 INFO (zkCallback-641-thread-1-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 609381 INFO (zkCallback-635-thread-2-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 609975 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/shard-3-001/cores/collection1
[junit4] 2> 609977 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/shard-3-001
[junit4] 2> 609978 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 609981 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3acf6dce{/_cn/ug,null,AVAILABLE}
[junit4] 2> 609981 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4b5c3fc8{HTTP/1.1,[http/1.1]}{127.0.0.1:60125}
[junit4] 2> 609981 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.e.j.s.Server Started @614771ms
[junit4] 2> 609981 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/tempDir-001/jetty3, solrconfig=solrconfig.xml, hostContext=/_cn/ug, hostPort=60125, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/shard-3-001/cores}
[junit4] 2> 609982 ERROR (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 609983 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 609983 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 609983 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 609983 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-01T07:11:49.011Z
[junit4] 2> 609989 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 609989 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/shard-3-001/solr.xml
[junit4] 2> 610004 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60096/solr
[junit4] 2> 610027 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [n:127.0.0.1:60125__cn%2Fug ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 610038 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [n:127.0.0.1:60125__cn%2Fug ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60125__cn%2Fug
[junit4] 2> 610042 INFO (zkCallback-625-thread-3-processing-n:127.0.0.1:60099__cn%2Fug) [n:127.0.0.1:60099__cn%2Fug ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 610044 INFO (zkCallback-629-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 610043 INFO (zkCallback-635-thread-2-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 610045 INFO (zkCallback-641-thread-1-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 610046 INFO (zkCallback-648-thread-1-processing-n:127.0.0.1:60125__cn%2Fug) [n:127.0.0.1:60125__cn%2Fug ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 610144 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [n:127.0.0.1:60125__cn%2Fug ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/shard-3-001/cores
[junit4] 2> 610144 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [n:127.0.0.1:60125__cn%2Fug ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 610151 INFO (OverseerStateUpdate-97382111737610244-127.0.0.1:60099__cn%2Fug-n_0000000000) [n:127.0.0.1:60099__cn%2Fug ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 610257 INFO (zkCallback-641-thread-1-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 610257 INFO (zkCallback-648-thread-1-processing-n:127.0.0.1:60125__cn%2Fug) [n:127.0.0.1:60125__cn%2Fug ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 610257 INFO (zkCallback-635-thread-2-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 610282 INFO (qtp974522949-3685) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:60120__cn%2Fug, coreNodeName=core_node2, onlyIfActiveCheckResult=false, nodeProps: core_node2:{"core":"collection1","base_url":"http://127.0.0.1:60120/_cn/ug","node_name":"127.0.0.1:60120__cn%2Fug","state":"recovering"}
[junit4] 2> 610282 INFO (qtp974522949-3685) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 610282 INFO (qtp974522949-3685) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:60120__cn%252Fug&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node2&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1001
[junit4] 2> 611181 INFO (coreLoadExecutor-2001-thread-1-processing-n:127.0.0.1:60125__cn%2Fug) [n:127.0.0.1:60125__cn%2Fug c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 611194 INFO (coreLoadExecutor-2001-thread-1-processing-n:127.0.0.1:60125__cn%2Fug) [n:127.0.0.1:60125__cn%2Fug c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 611293 WARN (coreLoadExecutor-2001-thread-1-processing-n:127.0.0.1:60125__cn%2Fug) [n:127.0.0.1:60125__cn%2Fug c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 611294 INFO (coreLoadExecutor-2001-thread-1-processing-n:127.0.0.1:60125__cn%2Fug) [n:127.0.0.1:60125__cn%2Fug c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 611315 INFO (coreLoadExecutor-2001-thread-1-processing-n:127.0.0.1:60125__cn%2Fug) [n:127.0.0.1:60125__cn%2Fug c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 611317 INFO (coreLoadExecutor-2001-thread-1-processing-n:127.0.0.1:60125__cn%2Fug) [n:127.0.0.1:60125__cn%2Fug c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/shard-3-001/cores/collection1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001/shard-3-001/cores/collection1/data/]
[junit4] 2> 611317 INFO (coreLoadExecutor-2001-thread-1-processing-n:127.0.0.1:60125__cn%2Fug) [n:127.0.0.1:60125__cn%2Fug c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7739f914
[junit4] 2> 611320 INFO (coreLoadExecutor-2001-thread-1-processing-n:127.0.0.1:60125__cn%2Fug) [n:127.0.0.1:60125__cn%2Fug c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=43, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=3.126953125, floorSegmentMB=0.302734375, forceMergeDeletesPctAllowed=20.08510998242944, segmentsPerTier=43.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.31395065872592465
[junit4] 2> 611348 WARN (coreLoadExecutor-2001-thread-1-processing-n:127.0.0.1:60125__cn%2Fug) [n:127.0.0.1:60125__cn%2Fug c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 611428 INFO (coreLoadExecutor-2001-thread-1-processing-n:127.0.0.1:60125__cn%2Fug) [n:127.0.0.1:60125__cn%2Fug c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 611428 INFO (coreLoadExecutor-2001-thread-1-processing-n:127.0.0.1:60125__cn%2Fug) [n:127.0.0.1:60125__cn%2Fug c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 611429 INFO (coreLoadExecutor-2001-thread-1-processing-n:127.0.0.1:60125__cn%2Fug) [n:127.0.0.1:60125__cn%2Fug c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 611429 INFO (coreLoadExecutor-2001-thread-1-processing-n:127.0.0.1:60125__cn%2Fug) [n:127.0.0.1:60125__cn%2Fug c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 611430 INFO (coreLoadExecutor-2001-thread-1-processing-n:127.0.0.1:60125__cn%2Fug) [n:127.0.0.1:60125__cn%2Fug c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@1f7ff436
[junit4] 2> 611432 INFO (coreLoadExecutor-2001-thread-1-processing-n:127.0.0.1:60125__cn%2Fug) [n:127.0.0.1:60125__cn%2Fug c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4e86d7eb[collection1] main]
[junit4] 2> 611434 INFO (coreLoadExecutor-2001-thread-1-processing-n:127.0.0.1:60125__cn%2Fug) [n:127.0.0.1:60125__cn%2Fug c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 611434 INFO (coreLoadExecutor-2001-thread-1-processing-n:127.0.0.1:60125__cn%2Fug) [n:127.0.0.1:60125__cn%2Fug c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 611435 INFO (coreLoadExecutor-2001-thread-1-processing-n:127.0.0.1:60125__cn%2Fug) [n:127.0.0.1:60125__cn%2Fug c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 611437 INFO (searcherExecutor-2002-thread-1-processing-n:127.0.0.1:60125__cn%2Fug x:collection1 c:collection1) [n:127.0.0.1:60125__cn%2Fug c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4e86d7eb[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 611438 INFO (coreLoadExecutor-2001-thread-1-processing-n:127.0.0.1:60125__cn%2Fug) [n:127.0.0.1:60125__cn%2Fug c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1558113797239996416
[junit4] 2> 611443 INFO (coreZkRegister-1996-thread-1-processing-n:127.0.0.1:60125__cn%2Fug x:collection1 c:collection1) [n:127.0.0.1:60125__cn%2Fug c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.ZkController Core needs to recover:collection1
[junit4] 2> 611444 INFO (updateExecutor-645-thread-1-processing-n:127.0.0.1:60125__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:60125__cn%2Fug c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 611444 INFO (recoveryExecutor-646-thread-1-processing-n:127.0.0.1:60125__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:60125__cn%2Fug c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 611445 INFO (recoveryExecutor-646-thread-1-processing-n:127.0.0.1:60125__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:60125__cn%2Fug c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 611445 INFO (recoveryExecutor-646-thread-1-processing-n:127.0.0.1:60125__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:60125__cn%2Fug c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1]
[junit4] 2> 611445 INFO (recoveryExecutor-646-thread-1-processing-n:127.0.0.1:60125__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:60125__cn%2Fug c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 611445 INFO (recoveryExecutor-646-thread-1-processing-n:127.0.0.1:60125__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:60125__cn%2Fug c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core [collection1] as recovering, leader is [http://127.0.0.1:60105/_cn/ug/collection1/] and I am [http://127.0.0.1:60125/_cn/ug/collection1/]
[junit4] 2> 611450 INFO (recoveryExecutor-646-thread-1-processing-n:127.0.0.1:60125__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:60125__cn%2Fug c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:60105/_cn/ug]; [WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:60125__cn%252Fug&coreNodeName=core_node3&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 611452 INFO (qtp974522949-3689) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 611453 INFO (qtp974522949-3689) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 (shard1 of collection1) have state: recovering
[junit4] 2> 611453 INFO (qtp974522949-3689) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:60125__cn%2Fug, coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: core_node3:{"core":"collection1","base_url":"http://127.0.0.1:60125/_cn/ug","node_name":"127.0.0.1:60125__cn%2Fug","state":"down"}
[junit4] 2> 611552 INFO (zkCallback-635-thread-2-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 611552 INFO (zkCallback-641-thread-1-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 611552 INFO (zkCallback-648-thread-1-processing-n:127.0.0.1:60125__cn%2Fug) [n:127.0.0.1:60125__cn%2Fug ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 611881 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 611881 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Wait for recoveries to finish - wait 30 for each attempt
[junit4] 2> 611881 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):30
[junit4] 2> 612453 INFO (qtp974522949-3689) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:60125__cn%2Fug, coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: core_node3:{"core":"collection1","base_url":"http://127.0.0.1:60125/_cn/ug","node_name":"127.0.0.1:60125__cn%2Fug","state":"recovering"}
[junit4] 2> 612453 INFO (qtp974522949-3689) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 612453 INFO (qtp974522949-3689) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:60125__cn%252Fug&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node3&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1001
[junit4] 2> 617293 INFO (recoveryExecutor-639-thread-1-processing-n:127.0.0.1:60120__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:60120__cn%2Fug c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:60105/_cn/ug/collection1/] - recoveringAfterStartup=[true]
[junit4] 2> 617293 INFO (recoveryExecutor-639-thread-1-processing-n:127.0.0.1:60120__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:60120__cn%2Fug c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.PeerSync PeerSync: core=collection1 url=http://127.0.0.1:60120/_cn/ug START replicas=[http://127.0.0.1:60105/_cn/ug/collection1/] nUpdates=1000
[junit4] 2> 617298 INFO (qtp974522949-3683) [n:127.0.0.1:60105__cn%2Fug c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:1.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 617298 INFO (qtp974522949-3683) [n:127.0.0.1:60105__cn%2Fug c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.S.Request [collection1] webapp=/_cn/ug path=/get params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 617300 INFO (recoveryExecutor-639-thread-1-processing-n:127.0.0.1:60120__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:60120__cn%2Fug c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:1.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 617300 INFO (recoveryExecutor-639-thread-1-processing-n:127.0.0.1:60120__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:60120__cn%2Fug c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.PeerSync We are already in sync. No need to do a PeerSync
[junit4] 2> 617300 INFO (recoveryExecutor-639-thread-1-processing-n:127.0.0.1:60120__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:60120__cn%2Fug c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 617300 INFO (recoveryExecutor-639-thread-1-processing-n:127.0.0.1:60120__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:60120__cn%2Fug c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 617302 INFO (recoveryExecutor-639-thread-1-processing-n:127.0.0.1:60120__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:60120__cn%2Fug c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 617302 INFO (recoveryExecutor-639-thread-1-processing-n:127.0.0.1:60120__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:60120__cn%2Fug c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy PeerSync stage of recovery was successful.
[junit4] 2> 617302 INFO (recoveryExecutor-639-thread-1-processing-n:127.0.0.1:60120__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:60120__cn%2Fug c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Replaying updates buffered during PeerSync.
[junit4] 2> 617302 INFO (recoveryExecutor-639-thread-1-processing-n:127.0.0.1:60120__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:60120__cn%2Fug c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy No replay needed.
[junit4] 2> 617302 INFO (recoveryExecutor-639-thread-1-processing-n:127.0.0.1:60120__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:60120__cn%2Fug c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Registering as Active after recovery.
[junit4] 2> 617309 INFO (zkCallback-641-thread-1-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 617309 INFO (zkCallback-635-thread-2-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 617309 INFO (zkCallback-648-thread-1-processing-n:127.0.0.1:60125__cn%2Fug) [n:127.0.0.1:60125__cn%2Fug ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 619461 INFO (recoveryExecutor-646-thread-1-processing-n:127.0.0.1:60125__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:60125__cn%2Fug c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:60105/_cn/ug/collection1/] - recoveringAfterStartup=[true]
[junit4] 2> 619461 INFO (recoveryExecutor-646-thread-1-processing-n:127.0.0.1:60125__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:60125__cn%2Fug c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.PeerSync PeerSync: core=collection1 url=http://127.0.0.1:60125/_cn/ug START replicas=[http://127.0.0.1:60105/_cn/ug/collection1/] nUpdates=1000
[junit4] 2> 619465 INFO (qtp974522949-3687) [n:127.0.0.1:60105__cn%2Fug c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 619465 INFO (qtp974522949-3687) [n:127.0.0.1:60105__cn%2Fug c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.S.Request [collection1] webapp=/_cn/ug path=/get params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 619467 INFO (recoveryExecutor-646-thread-1-processing-n:127.0.0.1:60125__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:60125__cn%2Fug c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:1.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 619467 INFO (recoveryExecutor-646-thread-1-processing-n:127.0.0.1:60125__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:60125__cn%2Fug c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.PeerSync We are already in sync. No need to do a PeerSync
[junit4] 2> 619467 INFO (recoveryExecutor-646-thread-1-processing-n:127.0.0.1:60125__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:60125__cn%2Fug c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 619467 INFO (recoveryExecutor-646-thread-1-processing-n:127.0.0.1:60125__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:60125__cn%2Fug c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 619467 INFO (recoveryExecutor-646-thread-1-processing-n:127.0.0.1:60125__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:60125__cn%2Fug c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 619467 INFO (recoveryExecutor-646-thread-1-processing-n:127.0.0.1:60125__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:60125__cn%2Fug c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy PeerSync stage of recovery was successful.
[junit4] 2> 619467 INFO (recoveryExecutor-646-thread-1-processing-n:127.0.0.1:60125__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:60125__cn%2Fug c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Replaying updates buffered during PeerSync.
[junit4] 2> 619467 INFO (recoveryExecutor-646-thread-1-processing-n:127.0.0.1:60125__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:60125__cn%2Fug c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy No replay needed.
[junit4] 2> 619467 INFO (recoveryExecutor-646-thread-1-processing-n:127.0.0.1:60125__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:60125__cn%2Fug c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Registering as Active after recovery.
[junit4] 2> 619473 INFO (zkCallback-648-thread-1-processing-n:127.0.0.1:60125__cn%2Fug) [n:127.0.0.1:60125__cn%2Fug ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 619473 INFO (zkCallback-635-thread-2-processing-n:127.0.0.1:60105__cn%2Fug) [n:127.0.0.1:60105__cn%2Fug ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 619473 INFO (zkCallback-641-thread-1-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 619942 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: collection1
[junit4] 2> 619946 INFO (qtp10715982-3645) [n:127.0.0.1:60099__cn%2Fug c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 619947 INFO (qtp10715982-3645) [n:127.0.0.1:60099__cn%2Fug c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 619948 INFO (qtp10715982-3645) [n:127.0.0.1:60099__cn%2Fug c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 619948 INFO (qtp10715982-3645) [n:127.0.0.1:60099__cn%2Fug c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/_cn/ug path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 1
[junit4] 2> 619959 INFO (qtp974522949-3689) [n:127.0.0.1:60105__cn%2Fug c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 619959 INFO (qtp974522949-3689) [n:127.0.0.1:60105__cn%2Fug c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 619959 INFO (qtp974522949-3689) [n:127.0.0.1:60105__cn%2Fug c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 619960 INFO (qtp974522949-3689) [n:127.0.0.1:60105__cn%2Fug c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/_cn/ug path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:60105/_cn/ug/collection1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 0
[junit4] 2> 619962 INFO (qtp394982622-3750) [n:127.0.0.1:60125__cn%2Fug c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 619964 INFO (qtp1740505651-3713) [n:127.0.0.1:60120__cn%2Fug c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 619966 INFO (qtp394982622-3750) [n:127.0.0.1:60125__cn%2Fug c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 619966 INFO (qtp1740505651-3713) [n:127.0.0.1:60120__cn%2Fug c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 619967 INFO (qtp394982622-3750) [n:127.0.0.1:60125__cn%2Fug c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 619967 INFO (qtp1740505651-3713) [n:127.0.0.1:60120__cn%2Fug c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 619968 INFO (qtp394982622-3750) [n:127.0.0.1:60125__cn%2Fug c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/_cn/ug path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:60105/_cn/ug/collection1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 5
[junit4] 2> 619969 INFO (qtp1740505651-3713) [n:127.0.0.1:60120__cn%2Fug c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/_cn/ug path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:60105/_cn/ug/collection1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 5
[junit4] 2> 619971 INFO (qtp974522949-3688) [n:127.0.0.1:60105__cn%2Fug c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/_cn/ug path=/update params={_stateVer_=collection1:10&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 19
[junit4] 2> 619975 INFO (qtp974522949-3690) [n:127.0.0.1:60105__cn%2Fug c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.S.Request [collection1] webapp=/_cn/ug path=/select params={q=*:*&distrib=false&tests=checkShardConsistency&rows=0&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 619978 INFO (qtp1740505651-3715) [n:127.0.0.1:60120__cn%2Fug c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.S.Request [collection1] webapp=/_cn/ug path=/select params={q=*:*&distrib=false&tests=checkShardConsistency&rows=0&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 619981 INFO (qtp394982622-3752) [n:127.0.0.1:60125__cn%2Fug c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.S.Request [collection1] webapp=/_cn/ug path=/select params={q=*:*&distrib=false&tests=checkShardConsistency&rows=0&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 621988 INFO (qtp10715982-3646) [n:127.0.0.1:60099__cn%2Fug c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/_cn/ug path=/update params={wt=javabin&version=2}{deleteByQuery=*:* (-1558113808298278912)} 0 4
[junit4] 2> 621998 INFO (qtp394982622-3752) [n:127.0.0.1:60125__cn%2Fug c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/_cn/ug path=/update params={update.distrib=FROMLEADER&_version_=-1558113808303521792&distrib.from=http://127.0.0.1:60105/_cn/ug/collection1/&wt=javabin&version=2}{deleteByQuery=*:* (-15581138083
[...truncated too long message...]
ore org.apache.solr.core.SolrCore@faf8dd2
[junit4] 2> 811114 WARN (recoveryExecutor-653-thread-1-processing-n:127.0.0.1:60120__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:60120__cn%2Fug c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Stopping recovery for core=[collection1] coreNodeName=[core_node2]
[junit4] 2> 811117 INFO (zkCallback-655-thread-3-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 811154 INFO (recoveryExecutor-653-thread-1-processing-n:127.0.0.1:60120__cn%2Fug x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:60120__cn%2Fug c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection1
[junit4] 2> 811155 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.Overseer Overseer (id=97382111737610258-127.0.0.1:60120__cn%2Fug-n_0000000004) closing
[junit4] 2> 811155 INFO (OverseerStateUpdate-97382111737610258-127.0.0.1:60120__cn%2Fug-n_0000000004) [n:127.0.0.1:60120__cn%2Fug ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:60120__cn%2Fug
[junit4] 2> 811157 WARN (zkCallback-655-thread-3-processing-n:127.0.0.1:60120__cn%2Fug) [n:127.0.0.1:60120__cn%2Fug ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 811158 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 811159 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@7c87b391{HTTP/1.1,[http/1.1]}{127.0.0.1:60120}
[junit4] 2> 811159 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2ef6c5ca{/_cn/ug,null,UNAVAILABLE}
[junit4] 2> 811160 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.ChaosMonkey monkey: stop shard! 60125
[junit4] 2> 811161 INFO (TEST-PeerSyncReplicationTest.test-seed#[5BBC5CEDFDB3B313]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:60096 60096
[junit4] 2> 811231 INFO (Thread-908) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:60096 60096
[junit4] 2> 813698 WARN (Thread-908) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/aliases.json
[junit4] 2> 6 /solr/clusterprops.json
[junit4] 2> 5 /solr/security.json
[junit4] 2> 5 /solr/configs/conf1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/clusterstate.json
[junit4] 2> 4 /solr/collections/collection1/state.json
[junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/97382111737610249-core_node1-n_0000000000
[junit4] 2> 2 /solr/overseer_elect/election/97382111737610249-127.0.0.1:60105__cn%2Fug-n_0000000001
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 209 /solr/overseer/collection-queue-work
[junit4] 2> 41 /solr/overseer/queue
[junit4] 2> 15 /solr/overseer/queue-work
[junit4] 2> 6 /solr/collections
[junit4] 2> 5 /solr/live_nodes
[junit4] 2>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=PeerSyncReplicationTest -Dtests.method=test -Dtests.seed=5BBC5CEDFDB3B313 -Dtests.slow=true -Dtests.locale=hi -Dtests.timezone=America/North_Dakota/Center -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] FAILURE 211s J1 | PeerSyncReplicationTest.test <<<
[junit4] > Throwable #1: java.lang.AssertionError: timeout waiting to see all nodes active
[junit4] > at __randomizedtesting.SeedInfo.seed([5BBC5CEDFDB3B313:D3E86337534FDEEB]:0)
[junit4] > at org.apache.solr.cloud.PeerSyncReplicationTest.waitTillNodesActive(PeerSyncReplicationTest.java:326)
[junit4] > at org.apache.solr.cloud.PeerSyncReplicationTest.bringUpDeadNodeAndEnsureNoReplication(PeerSyncReplicationTest.java:277)
[junit4] > at org.apache.solr.cloud.PeerSyncReplicationTest.forceNodeFailureAndDoPeerSync(PeerSyncReplicationTest.java:259)
[junit4] > at org.apache.solr.cloud.PeerSyncReplicationTest.test(PeerSyncReplicationTest.java:138)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 813702 INFO (SUITE-PeerSyncReplicationTest-seed#[5BBC5CEDFDB3B313]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_5BBC5CEDFDB3B313-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {other_tl1=BlockTreeOrds(blocksize=128), range_facet_l_dv=PostingsFormat(name=Memory), rnd_s=PostingsFormat(name=Memory), multiDefault=PostingsFormat(name=Memory), a_t=BlockTreeOrds(blocksize=128), intDefault=BlockTreeOrds(blocksize=128), a_i1=BlockTreeOrds(blocksize=128), id=PostingsFormat(name=Memory), range_facet_i_dv=BlockTreeOrds(blocksize=128), text=PostingsFormat(name=Direct), range_facet_l=BlockTreeOrds(blocksize=128), timestamp=BlockTreeOrds(blocksize=128)}, docValues:{other_tl1=DocValuesFormat(name=Asserting), range_facet_l_dv=DocValuesFormat(name=Lucene70), rnd_s=DocValuesFormat(name=Direct), multiDefault=DocValuesFormat(name=Direct), intDefault=DocValuesFormat(name=Asserting), a_i1=DocValuesFormat(name=Asserting), range_facet_l=DocValuesFormat(name=Asserting), _version_=DocValuesFormat(name=Asserting), a_t=DocValuesFormat(name=Asserting), range_facet_i_dv=DocValuesFormat(name=Asserting), id=DocValuesFormat(name=Lucene70), text=DocValuesFormat(name=Memory), intDvoDefault=DocValuesFormat(name=Memory), timestamp=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=1469, maxMBSortInHeap=5.802314970222217, sim=RandomSimilarity(queryNorm=false): {}, locale=hi, timezone=America/North_Dakota/Center
[junit4] 2> NOTE: Mac OS X 10.11.6 x86_64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=3,threads=1,free=71901456,total=254242816
[junit4] 2> NOTE: All tests run in this JVM: [DistributedFacetPivotSmallAdvancedTest, TestSystemIdResolver, TestPushWriter, ReplicationFactorTest, ZkSolrClientTest, TestJsonRequest, BadComponentTest, NotRequiredUniqueKeyTest, TestSchemalessBufferedUpdates, LukeRequestHandlerTest, TestLegacyNumericRangeQueryBuilder, TestAuthorizationFramework, BasicZkTest, TestSolr4Spatial2, HdfsBasicDistributedZkTest, DistanceUnitsTest, TestRebalanceLeaders, TestZkChroot, TestDocSet, TestCoreAdminApis, HLLSerializationTest, DistributedTermsComponentTest, SuggesterTSTTest, RollingRestartTest, LoggingHandlerTest, VersionInfoTest, BadIndexSchemaTest, SaslZkACLProviderTest, TestManagedSchemaAPI, SimpleMLTQParserTest, DistributedFacetPivotLargeTest, TestJettySolrRunner, UpdateParamsTest, TestRawTransformer, TestObjectReleaseTracker, CoreAdminHandlerTest, TestLMJelinekMercerSimilarityFactory, BasicDistributedZk2Test, TestConfig, TestTrie, TestSolrXml, PeerSyncWithIndexFingerprintCachingTest, TestConfigSetsAPIZkFailure, DefaultValueUpdateProcessorTest, FileBasedSpellCheckerTest, TestLRUStatsCache, TestComponentsName, TestSimpleTrackingShardHandler, BlockDirectoryTest, TestReqParamsAPI, TestWriterPerf, InfoHandlerTest, SecurityConfHandlerTest, TestAnalyzedSuggestions, DistanceFunctionTest, TestSolrDeletionPolicy2, StatsComponentTest, TestPivotHelperCode, HighlighterConfigTest, HdfsRecoveryZkTest, SolrGangliaReporterTest, EchoParamsTest, TestInitParams, ResponseLogComponentTest, SliceStateTest, SpellCheckComponentTest, URLClassifyProcessorTest, TestCloudInspectUtil, TestMacroExpander, TestHighFrequencyDictionaryFactory, TestSolrCloudWithSecureImpersonation, TestHdfsUpdateLog, LeaderElectionIntegrationTest, ConcurrentDeleteAndCreateCollectionTest, TestConfigSetImmutable, TestUniqueKeyFieldResource, PeerSyncReplicationTest]
[junit4] Completed [121/691 (1!)] on J1 in 211.29s, 1 test, 1 failure <<< FAILURES!
[...truncated 1852 lines...]
[junit4] JVM J1: stdout was not empty, see: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/temp/junit4-J1-20170201_070134_1257393734475254523071.sysout
[junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: Java heap space
[junit4] Dumping heap to /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/heapdumps/java_pid33191.hprof ...
[junit4] Heap dump file created [99923635 bytes in 3.977 secs]
[junit4] <<< JVM J1: EOF ----
[...truncated 11657 lines...]
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/build.xml:775: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/build.xml:727: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid33191.hprof
Total time: 105 minutes 26 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Re: [JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build #
3809 - Still Unstable!
Posted by Steve Rowe <sa...@gmail.com>.
Uwe,
Too many open files - you maybe need to configure the macOS VM?:
> On Jan 31, 2017, at 4:53 PM, Policeman Jenkins Server <je...@thetaphi.de> wrote:
>
> Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3809/
[…]
> FAILED: org.apache.solr.cloud.SimpleCollectionCreateDeleteTest.test
>
> Error Message:
> java.io.IOException: Couldn't instantiate org.apache.zookeeper.ClientCnxnSocketNIO
>
> Stack Trace:
> org.apache.solr.common.SolrException: java.io.IOException: Couldn't instantiate org.apache.zookeeper.ClientCnxnSocketNIO
> at __randomizedtesting.SeedInfo.seed([321AC469C3E8746B:BA4EFBB36D141993]:0)
> at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:170)
[…]
> Caused by: java.io.IOException: Too many open files
> at sun.nio.ch.IOUtil.makePipe(Native Method)
> at sun.nio.ch.KQueueSelectorImpl.<init>(KQueueSelectorImpl.java:84)
> at sun.nio.ch.KQueueSelectorProvider.openSelector(KQueueSelectorProvider.java:42)
> at java.nio.channels.Selector.open(Selector.java:227)
> at org.apache.zookeeper.ClientCnxnSocketNIO.<init>(ClientCnxnSocketNIO.java:43)
> at sun.reflect.GeneratedConstructorAccessor218.newInstance(Unknown Source)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at java.lang.Class.newInstance(Class.java:442)
> at org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:1779)
> ... 43 more
--
Steve
www.lucidworks.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org