You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2012/11/01 02:00:10 UTC

[JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 602 - Still Failing

Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/602/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.AnalysisAfterCoreReloadTest

Error Message:
4 threads leaked from SUITE scope at org.apache.solr.AnalysisAfterCoreReloadTest:     1) Thread[id=156, name=qtp337577407-156 Acceptor1 SelectChannelConnector@0.0.0.0:14453, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=155, name=qtp337577407-155 Acceptor0 SelectChannelConnector@0.0.0.0:14453, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=158, name=qtp337577407-158 Acceptor3 SelectChannelConnector@0.0.0.0:14453, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=157, name=qtp337577407-157 Acceptor2 SelectChannelConnector@0.0.0.0:14453, state=RUNNABLE, group=TGRP-AnalysisAfterCoreReloadTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.AnalysisAfterCoreReloadTest: 
   1) Thread[id=156, name=qtp337577407-156 Acceptor1 SelectChannelConnector@0.0.0.0:14453, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=155, name=qtp337577407-155 Acceptor0 SelectChannelConnector@0.0.0.0:14453, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=158, name=qtp337577407-158 Acceptor3 SelectChannelConnector@0.0.0.0:14453, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=157, name=qtp337577407-157 Acceptor2 SelectChannelConnector@0.0.0.0:14453, state=RUNNABLE, group=TGRP-AnalysisAfterCoreReloadTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([6FAC95058473C9E4]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.AnalysisAfterCoreReloadTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=156, name=qtp337577407-156 Acceptor1 SelectChannelConnector@0.0.0.0:14453, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=155, name=qtp337577407-155 Acceptor0 SelectChannelConnector@0.0.0.0:14453, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=158, name=qtp337577407-158 Acceptor3 SelectChannelConnector@0.0.0.0:14453, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=157, name=qtp337577407-157 Acceptor2 SelectChannelConnector@0.0.0.0:14453, state=RUNNABLE, group=TGRP-AnalysisAfterCoreReloadTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=156, name=qtp337577407-156 Acceptor1 SelectChannelConnector@0.0.0.0:14453, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=155, name=qtp337577407-155 Acceptor0 SelectChannelConnector@0.0.0.0:14453, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=158, name=qtp337577407-158 Acceptor3 SelectChannelConnector@0.0.0.0:14453, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=157, name=qtp337577407-157 Acceptor2 SelectChannelConnector@0.0.0.0:14453, state=RUNNABLE, group=TGRP-AnalysisAfterCoreReloadTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([6FAC95058473C9E4]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.component.TermVectorComponentDistributedTest

Error Message:
36 threads leaked from SUITE scope at org.apache.solr.handler.component.TermVectorComponentDistributedTest:     1) Thread[id=227, name=qtp537519666-227 Acceptor3 SelectChannelConnector@0.0.0.0:14355, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=254, name=qtp1915594299-254 Acceptor1 SelectChannelConnector@0.0.0.0:14377, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=292, name=qtp1688405571-292 Acceptor3 SelectChannelConnector@0.0.0.0:14394, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=226, name=qtp537519666-226 Acceptor2 SelectChannelConnector@0.0.0.0:14355, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=212, name=qtp1681108815-212 Acceptor2 SelectChannelConnector@0.0.0.0:14349, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=269, name=qtp325763057-269 Acceptor1 SelectChannelConnector@0.0.0.0:14380, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=289, name=qtp1688405571-289 Acceptor0 SelectChannelConnector@0.0.0.0:14394, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=331, name=qtp1761008431-331 Acceptor0 SelectChannelConnector@0.0.0.0:14400, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=225, name=qtp537519666-225 Acceptor1 SelectChannelConnector@0.0.0.0:14355, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=270, name=qtp325763057-270 Acceptor2 SelectChannelConnector@0.0.0.0:14380, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=241, name=qtp1430523914-241 Acceptor0 SelectChannelConnector@0.0.0.0:14374, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=319, name=qtp126023584-319 Acceptor2 SelectChannelConnector@0.0.0.0:14398, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=320, name=qtp126023584-320 Acceptor3 SelectChannelConnector@0.0.0.0:14398, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=306, name=qtp1129948032-306 Acceptor3 SelectChannelConnector@0.0.0.0:14396, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=255, name=qtp1915594299-255 Acceptor2 SelectChannelConnector@0.0.0.0:14377, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=290, name=qtp1688405571-290 Acceptor1 SelectChannelConnector@0.0.0.0:14394, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=244, name=qtp1430523914-244 Acceptor3 SelectChannelConnector@0.0.0.0:14374, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=243, name=qtp1430523914-243 Acceptor2 SelectChannelConnector@0.0.0.0:14374, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=334, name=qtp1761008431-334 Acceptor3 SelectChannelConnector@0.0.0.0:14400, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=291, name=qtp1688405571-291 Acceptor2 SelectChannelConnector@0.0.0.0:14394, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   21) Thread[id=256, name=qtp1915594299-256 Acceptor3 SelectChannelConnector@0.0.0.0:14377, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   22) Thread[id=224, name=qtp537519666-224 Acceptor0 SelectChannelConnector@0.0.0.0:14355, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   23) Thread[id=317, name=qtp126023584-317 Acceptor0 SelectChannelConnector@0.0.0.0:14398, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   24) Thread[id=303, name=qtp1129948032-303 Acceptor0 SelectChannelConnector@0.0.0.0:14396, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   25) Thread[id=318, name=qtp126023584-318 Acceptor1 SelectChannelConnector@0.0.0.0:14398, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   26) Thread[id=333, name=qtp1761008431-333 Acceptor2 SelectChannelConnector@0.0.0.0:14400, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   27) Thread[id=271, name=qtp325763057-271 Acceptor3 SelectChannelConnector@0.0.0.0:14380, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   28) Thread[id=268, name=qtp325763057-268 Acceptor0 SelectChannelConnector@0.0.0.0:14380, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   29) Thread[id=213, name=qtp1681108815-213 Acceptor3 SelectChannelConnector@0.0.0.0:14349, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   30) Thread[id=253, name=qtp1915594299-253 Acceptor0 SelectChannelConnector@0.0.0.0:14377, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   31) Thread[id=332, name=qtp1761008431-332 Acceptor1 SelectChannelConnector@0.0.0.0:14400, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   32) Thread[id=305, name=qtp1129948032-305 Acceptor2 SelectChannelConnector@0.0.0.0:14396, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   33) Thread[id=304, name=qtp1129948032-304 Acceptor1 SelectChannelConnector@0.0.0.0:14396, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   34) Thread[id=242, name=qtp1430523914-242 Acceptor1 SelectChannelConnector@0.0.0.0:14374, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   35) Thread[id=210, name=qtp1681108815-210 Acceptor0 SelectChannelConnector@0.0.0.0:14349, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   36) Thread[id=211, name=qtp1681108815-211 Acceptor1 SelectChannelConnector@0.0.0.0:14349, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 36 threads leaked from SUITE scope at org.apache.solr.handler.component.TermVectorComponentDistributedTest: 
   1) Thread[id=227, name=qtp537519666-227 Acceptor3 SelectChannelConnector@0.0.0.0:14355, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=254, name=qtp1915594299-254 Acceptor1 SelectChannelConnector@0.0.0.0:14377, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=292, name=qtp1688405571-292 Acceptor3 SelectChannelConnector@0.0.0.0:14394, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=226, name=qtp537519666-226 Acceptor2 SelectChannelConnector@0.0.0.0:14355, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=212, name=qtp1681108815-212 Acceptor2 SelectChannelConnector@0.0.0.0:14349, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=269, name=qtp325763057-269 Acceptor1 SelectChannelConnector@0.0.0.0:14380, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=289, name=qtp1688405571-289 Acceptor0 SelectChannelConnector@0.0.0.0:14394, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=331, name=qtp1761008431-331 Acceptor0 SelectChannelConnector@0.0.0.0:14400, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=225, name=qtp537519666-225 Acceptor1 SelectChannelConnector@0.0.0.0:14355, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=270, name=qtp325763057-270 Acceptor2 SelectChannelConnector@0.0.0.0:14380, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=241, name=qtp1430523914-241 Acceptor0 SelectChannelConnector@0.0.0.0:14374, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=319, name=qtp126023584-319 Acceptor2 SelectChannelConnector@0.0.0.0:14398, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=320, name=qtp126023584-320 Acceptor3 SelectChannelConnector@0.0.0.0:14398, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=306, name=qtp1129948032-306 Acceptor3 SelectChannelConnector@0.0.0.0:14396, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=255, name=qtp1915594299-255 Acceptor2 SelectChannelConnector@0.0.0.0:14377, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=290, name=qtp1688405571-290 Acceptor1 SelectChannelConnector@0.0.0.0:14394, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=244, name=qtp1430523914-244 Acceptor3 SelectChannelConnector@0.0.0.0:14374, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=243, name=qtp1430523914-243 Acceptor2 SelectChannelConnector@0.0.0.0:14374, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=334, name=qtp1761008431-334 Acceptor3 SelectChannelConnector@0.0.0.0:14400, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=291, name=qtp1688405571-291 Acceptor2 SelectChannelConnector@0.0.0.0:14394, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  21) Thread[id=256, name=qtp1915594299-256 Acceptor3 SelectChannelConnector@0.0.0.0:14377, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  22) Thread[id=224, name=qtp537519666-224 Acceptor0 SelectChannelConnector@0.0.0.0:14355, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  23) Thread[id=317, name=qtp126023584-317 Acceptor0 SelectChannelConnector@0.0.0.0:14398, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  24) Thread[id=303, name=qtp1129948032-303 Acceptor0 SelectChannelConnector@0.0.0.0:14396, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  25) Thread[id=318, name=qtp126023584-318 Acceptor1 SelectChannelConnector@0.0.0.0:14398, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  26) Thread[id=333, name=qtp1761008431-333 Acceptor2 SelectChannelConnector@0.0.0.0:14400, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  27) Thread[id=271, name=qtp325763057-271 Acceptor3 SelectChannelConnector@0.0.0.0:14380, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  28) Thread[id=268, name=qtp325763057-268 Acceptor0 SelectChannelConnector@0.0.0.0:14380, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  29) Thread[id=213, name=qtp1681108815-213 Acceptor3 SelectChannelConnector@0.0.0.0:14349, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  30) Thread[id=253, name=qtp1915594299-253 Acceptor0 SelectChannelConnector@0.0.0.0:14377, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  31) Thread[id=332, name=qtp1761008431-332 Acceptor1 SelectChannelConnector@0.0.0.0:14400, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  32) Thread[id=305, name=qtp1129948032-305 Acceptor2 SelectChannelConnector@0.0.0.0:14396, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  33) Thread[id=304, name=qtp1129948032-304 Acceptor1 SelectChannelConnector@0.0.0.0:14396, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  34) Thread[id=242, name=qtp1430523914-242 Acceptor1 SelectChannelConnector@0.0.0.0:14374, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  35) Thread[id=210, name=qtp1681108815-210 Acceptor0 SelectChannelConnector@0.0.0.0:14349, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  36) Thread[id=211, name=qtp1681108815-211 Acceptor1 SelectChannelConnector@0.0.0.0:14349, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([6FAC95058473C9E4]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.component.TermVectorComponentDistributedTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=227, name=qtp537519666-227 Acceptor3 SelectChannelConnector@0.0.0.0:14355, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=254, name=qtp1915594299-254 Acceptor1 SelectChannelConnector@0.0.0.0:14377, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=292, name=qtp1688405571-292 Acceptor3 SelectChannelConnector@0.0.0.0:14394, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=226, name=qtp537519666-226 Acceptor2 SelectChannelConnector@0.0.0.0:14355, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=212, name=qtp1681108815-212 Acceptor2 SelectChannelConnector@0.0.0.0:14349, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=269, name=qtp325763057-269 Acceptor1 SelectChannelConnector@0.0.0.0:14380, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=289, name=qtp1688405571-289 Acceptor0 SelectChannelConnector@0.0.0.0:14394, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=331, name=qtp1761008431-331 Acceptor0 SelectChannelConnector@0.0.0.0:14400, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=225, name=qtp537519666-225 Acceptor1 SelectChannelConnector@0.0.0.0:14355, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=270, name=qtp325763057-270 Acceptor2 SelectChannelConnector@0.0.0.0:14380, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=241, name=qtp1430523914-241 Acceptor0 SelectChannelConnector@0.0.0.0:14374, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=319, name=qtp126023584-319 Acceptor2 SelectChannelConnector@0.0.0.0:14398, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=320, name=qtp126023584-320 Acceptor3 SelectChannelConnector@0.0.0.0:14398, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=306, name=qtp1129948032-306 Acceptor3 SelectChannelConnector@0.0.0.0:14396, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=255, name=qtp1915594299-255 Acceptor2 SelectChannelConnector@0.0.0.0:14377, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=290, name=qtp1688405571-290 Acceptor1 SelectChannelConnector@0.0.0.0:14394, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=243, name=qtp1430523914-243 Acceptor2 SelectChannelConnector@0.0.0.0:14374, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=244, name=qtp1430523914-244 Acceptor3 SelectChannelConnector@0.0.0.0:14374, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=291, name=qtp1688405571-291 Acceptor2 SelectChannelConnector@0.0.0.0:14394, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=334, name=qtp1761008431-334 Acceptor3 SelectChannelConnector@0.0.0.0:14400, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   21) Thread[id=224, name=qtp537519666-224 Acceptor0 SelectChannelConnector@0.0.0.0:14355, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   22) Thread[id=256, name=qtp1915594299-256 Acceptor3 SelectChannelConnector@0.0.0.0:14377, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   23) Thread[id=317, name=qtp126023584-317 Acceptor0 SelectChannelConnector@0.0.0.0:14398, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   24) Thread[id=303, name=qtp1129948032-303 Acceptor0 SelectChannelConnector@0.0.0.0:14396, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   25) Thread[id=318, name=qtp126023584-318 Acceptor1 SelectChannelConnector@0.0.0.0:14398, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   26) Thread[id=333, name=qtp1761008431-333 Acceptor2 SelectChannelConnector@0.0.0.0:14400, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   27) Thread[id=271, name=qtp325763057-271 Acceptor3 SelectChannelConnector@0.0.0.0:14380, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   28) Thread[id=268, name=qtp325763057-268 Acceptor0 SelectChannelConnector@0.0.0.0:14380, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   29) Thread[id=213, name=qtp1681108815-213 Acceptor3 SelectChannelConnector@0.0.0.0:14349, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   30) Thread[id=253, name=qtp1915594299-253 Acceptor0 SelectChannelConnector@0.0.0.0:14377, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   31) Thread[id=332, name=qtp1761008431-332 Acceptor1 SelectChannelConnector@0.0.0.0:14400, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   32) Thread[id=305, name=qtp1129948032-305 Acceptor2 SelectChannelConnector@0.0.0.0:14396, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   33) Thread[id=304, name=qtp1129948032-304 Acceptor1 SelectChannelConnector@0.0.0.0:14396, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   34) Thread[id=242, name=qtp1430523914-242 Acceptor1 SelectChannelConnector@0.0.0.0:14374, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   35) Thread[id=210, name=qtp1681108815-210 Acceptor0 SelectChannelConnector@0.0.0.0:14349, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   36) Thread[id=211, name=qtp1681108815-211 Acceptor1 SelectChannelConnector@0.0.0.0:14349, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=227, name=qtp537519666-227 Acceptor3 SelectChannelConnector@0.0.0.0:14355, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=254, name=qtp1915594299-254 Acceptor1 SelectChannelConnector@0.0.0.0:14377, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=292, name=qtp1688405571-292 Acceptor3 SelectChannelConnector@0.0.0.0:14394, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=226, name=qtp537519666-226 Acceptor2 SelectChannelConnector@0.0.0.0:14355, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=212, name=qtp1681108815-212 Acceptor2 SelectChannelConnector@0.0.0.0:14349, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=269, name=qtp325763057-269 Acceptor1 SelectChannelConnector@0.0.0.0:14380, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=289, name=qtp1688405571-289 Acceptor0 SelectChannelConnector@0.0.0.0:14394, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=331, name=qtp1761008431-331 Acceptor0 SelectChannelConnector@0.0.0.0:14400, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=225, name=qtp537519666-225 Acceptor1 SelectChannelConnector@0.0.0.0:14355, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=270, name=qtp325763057-270 Acceptor2 SelectChannelConnector@0.0.0.0:14380, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=241, name=qtp1430523914-241 Acceptor0 SelectChannelConnector@0.0.0.0:14374, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=319, name=qtp126023584-319 Acceptor2 SelectChannelConnector@0.0.0.0:14398, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=320, name=qtp126023584-320 Acceptor3 SelectChannelConnector@0.0.0.0:14398, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=306, name=qtp1129948032-306 Acceptor3 SelectChannelConnector@0.0.0.0:14396, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=255, name=qtp1915594299-255 Acceptor2 SelectChannelConnector@0.0.0.0:14377, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=290, name=qtp1688405571-290 Acceptor1 SelectChannelConnector@0.0.0.0:14394, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=243, name=qtp1430523914-243 Acceptor2 SelectChannelConnector@0.0.0.0:14374, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=244, name=qtp1430523914-244 Acceptor3 SelectChannelConnector@0.0.0.0:14374, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=291, name=qtp1688405571-291 Acceptor2 SelectChannelConnector@0.0.0.0:14394, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=334, name=qtp1761008431-334 Acceptor3 SelectChannelConnector@0.0.0.0:14400, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  21) Thread[id=224, name=qtp537519666-224 Acceptor0 SelectChannelConnector@0.0.0.0:14355, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  22) Thread[id=256, name=qtp1915594299-256 Acceptor3 SelectChannelConnector@0.0.0.0:14377, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  23) Thread[id=317, name=qtp126023584-317 Acceptor0 SelectChannelConnector@0.0.0.0:14398, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  24) Thread[id=303, name=qtp1129948032-303 Acceptor0 SelectChannelConnector@0.0.0.0:14396, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  25) Thread[id=318, name=qtp126023584-318 Acceptor1 SelectChannelConnector@0.0.0.0:14398, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  26) Thread[id=333, name=qtp1761008431-333 Acceptor2 SelectChannelConnector@0.0.0.0:14400, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  27) Thread[id=271, name=qtp325763057-271 Acceptor3 SelectChannelConnector@0.0.0.0:14380, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  28) Thread[id=268, name=qtp325763057-268 Acceptor0 SelectChannelConnector@0.0.0.0:14380, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  29) Thread[id=213, name=qtp1681108815-213 Acceptor3 SelectChannelConnector@0.0.0.0:14349, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  30) Thread[id=253, name=qtp1915594299-253 Acceptor0 SelectChannelConnector@0.0.0.0:14377, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  31) Thread[id=332, name=qtp1761008431-332 Acceptor1 SelectChannelConnector@0.0.0.0:14400, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  32) Thread[id=305, name=qtp1129948032-305 Acceptor2 SelectChannelConnector@0.0.0.0:14396, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  33) Thread[id=304, name=qtp1129948032-304 Acceptor1 SelectChannelConnector@0.0.0.0:14396, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  34) Thread[id=242, name=qtp1430523914-242 Acceptor1 SelectChannelConnector@0.0.0.0:14374, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  35) Thread[id=210, name=qtp1681108815-210 Acceptor0 SelectChannelConnector@0.0.0.0:14349, state=RUNNABLE, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  36) Thread[id=211, name=qtp1681108815-211 Acceptor1 SelectChannelConnector@0.0.0.0:14349, state=BLOCKED, group=TGRP-TermVectorComponentDistributedTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([6FAC95058473C9E4]:0)




Build Log:
[...truncated 8400 lines...]
[junit4:junit4] Suite: org.apache.solr.handler.component.TermVectorComponentDistributedTest
[junit4:junit4]   2> 9 T203 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> 637 T203 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 961 T203 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:14349
[junit4:junit4]   2> 1027 T203 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1028 T203 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 1028 T203 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 1090 T203 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1093 T203 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1094 T203 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 1094 T203 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 1095 T203 oasc.CoreContainer.<init> New CoreContainer 1848447389
[junit4:junit4]   2> 1095 T203 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 1096 T203 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 1139 T203 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 1164 T203 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 1165 T203 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 1166 T203 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 1167 T203 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 1314 T203 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 1440 T203 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1441 T203 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 1503 T203 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 2925 T203 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 2943 T203 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 2947 T203 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 3214 T203 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 3228 T203 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 3233 T203 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 3239 T203 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 3242 T203 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.handler.component.TermVectorComponentDistributedTest-1351731529440/control/data/
[junit4:junit4]   2> 3245 T203 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3b430892
[junit4:junit4]   2> 3251 T203 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.handler.component.TermVectorComponentDistributedTest-1351731529440/control/data/index/
[junit4:junit4]   2> 3252 T203 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.handler.component.TermVectorComponentDistributedTest-1351731529440/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 3252 T203 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.handler.component.TermVectorComponentDistributedTest-1351731529440/control/data/index forceNew:false
[junit4:junit4]   2> 3275 T203 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@153e6d0b lockFactory=org.apache.lucene.store.NativeFSLockFactory@7e768122),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 3275 T203 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 3306 T203 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 3439 T203 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 3440 T203 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 3441 T203 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 3447 T203 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 3543 T203 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 3543 T203 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 3544 T203 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 3545 T203 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 3546 T203 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 3596 T203 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 3599 T203 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 3599 T203 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 3600 T203 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 3601 T203 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 3601 T203 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 3602 T203 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 3603 T203 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3604 T203 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3605 T203 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3606 T203 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3607 T203 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3608 T203 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3609 T203 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3627 T203 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 3645 T203 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 3658 T203 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 3663 T203 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 3666 T203 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [THROW.ERROR.ON.ADD.UPDATEPROCESSOR.JS, SOLRCONFIG-HIGHLIGHT.XML, SCHEMA-REQUIRED-FIELDS.XML, BAD-SCHEMA-DUP-DYNAMICFIELD.XML, SCHEMA-MINIMAL.XML, TRIVIAL.UPDATEPROCESSOR1.JS, SCHEMA-REPLICATION2.XML, SOLRCONFIG-CACHING.XML, SOLRCONFIG-REPEATER.XML, SOLRCONFIG-MERGEPOLICY.XML, BAD-SCHEMA-NONTEXT-ANALYZER.XML, CURRENCY.XML, SOLRCONFIG-TLOG.XML, SOLRCONFIG-MASTER.XML, BAD-SCHEMA-UNIQUEKEY-MULTIVALUED.XML, SCHEMA11.XML, SOLRCONFIG-BASIC.XML, DA_COMPOUNDDICTIONARY.TXT, MISSLEADING.EXTENSION.UPDATEPROCESSOR.JS.TXT, SCHEMA-COPYFIELD-TEST.XML, SOLRCONFIG-SLAVE.XML, SOLRCONFIG-PROPINJECT-INDEXDEFAULT.XML, ELEVATE.XML, TRIVIAL.UPDATEPROCESSOR0.JS, BAD-SOLRCONFIG-BOGUS-SCRIPTENGINE-NAME.XML, SCHEMA-CHARFILTERS.XML, SCHEMA-IB.XML, SOLRCONFIG-QUERYSENDER.XML, DA_UTF8.XML, SCHEMA-REPLICATION1.XML, SOLRCONFIG-SNIPPET-PROCESSOR.XML, CONDITIONAL.UPDATEPROCESSOR.JS, MISSING.FUNCTIONS.UPDATEPROCESSOR.JS, HYPHENATION.DTD, SOLRCONFIG-ENABLEPLUGIN.XML, STEMDICT.TXT, SCHEMA-PHRASESUGGEST.XML, HUNSPELL-TEST.AFF, SCHEMA-SNIPPET-TYPE.XML, STOPTYPES-1.TXT, STOPWORDSWRONGENCODING.TXT, SCHEMA-NUMERIC.XML, SOLRCONFIG-TRANSFORMERS.XML, SOLRCONFIG-PROPINJECT.XML, BAD-SCHEMA-NOT-INDEXED-BUT-TF.XML, SOLRCONFIG-SIMPLELOCK.XML, WDFTYPES.TXT, STOPTYPES-2.TXT, SCHEMA-REVERSED.XML, BAD-SCHEMA-CURRENCY-FT-MULTIVALUED.XML, SOLRCONFIG-SPELLCHECKCOMPONENT.XML, SCHEMA-DFR.XML, SOLRCONFIG-PHRASESUGGEST.XML, BAD-SCHEMA-CURRENCY-MULTIVALUED.XML, BAD-SCHEMA-NOT-INDEXED-BUT-POS.XML, KEEP-1.TXT, SCHEMA-SPATIAL.XML, OPEN-EXCHANGE-RATES.JSON, STOPWITHBOM.TXT, SOLRCONFIG-SPELLCHECKER.XML, SCHEMA-BINARYFIELD.XML, SOLRCONFIG-UPDATE-PROCESSOR-CHAINS.XML, BAD-SCHEMA-OMIT-TF-BUT-NOT-POS.XML, BAD-SCHEMA-DUP-FIELDTYPE.XML, SCHEMA-XINCLUDE.XML, SOLRCONFIG-MASTER1.XML, SYNONYMS.TXT, BAD-SCHEMA-CURRENCY-DYNAMIC-MULTIVALUED.XML, SCHEMA.XML, SCHEMA_CODEC.XML, SOLRCONFIG-SOLR-749.XML, SOLRCONFIG-MASTER1-KEEPONEBACKUP.XML, STOP-2.TXT, SOLRCONFIG-FUNCTIONQUERY.XML, SCHEMA-LMDIRICHLET.XML, SOLRCONFIG-TERMINDEX.XML, SOLRCONFIG-ELEVATE.XML, SCHEMA-FOLDING.XML, STOPWORDS.TXT, SCHEMA-STOP-KEEP.XML, BAD-SCHEMA-NOT-INDEXED-BUT-NORMS.XML, SOLRCONFIG-SOLCOREPROPERTIES.XML, STOP-1.TXT, SOLRCONFIG-MASTER2.XML, SCHEMA-SPELLCHECKER.XML, SOLRCONFIG-RESPONSE-LOG-COMPONENT.XML, SOLRCONFIG-LAZYWRITER.XML, SCHEMA-LUCENEMATCHVERSION.XML, FRENCHARTICLES.TXT, BAD-MP-SOLRCONFIG.XML, SCHEMA15.XML, SOLRCONFIG-REQHANDLER.INCL, SCHEMASURROUND.XML, SCHEMA-COLLATEFILTER.XML, SOLRCONFIG-MASTER3.XML, HUNSPELL-TEST.DIC, BAD-SCHEMA-CODEC-GLOBAL-VS-FT-MISMATCH.XML, SOLRCONFIG-XINCLUDE.XML, SOLRCONFIG-SLAVE1.XML, SOLRCONFIG-DELPOLICY1.XML, SCHEMA-SIM.XML, SCHEMA-SNIPPET-FIELD.XML, SCHEMA-COLLATE.XML, STOP-SNOWBALL.TXT, BAD-SCHEMA-SIM-GLOBAL-VS-FT-MISMATCH.XML, PROTWORDS.TXT, SCHEMA-TRIE.XML, SOLRCONFIG_CODEC.XML, BAD-SOLRCONFIG-INVALID-SCRIPTFILE.XML, JASUGGEST.TXT, SCHEMA-TFIDF.XML, SOLRCONFIG-SCRIPT-UPDATEPROCESSOR.XML, SCHEMA-LMJELINEKMERCER.XML, PHRASESUGGEST.TXT, BAD-SOLRCONFIG-MISSING-SCRIPTFILE.XML, SOLRCONFIG-BASIC-LUCENEVERSION31.XML, BAD-SCHEMA-UNIQUEKEY-USES-DEFAULT.XML, OLD_SYNONYMS.TXT, SOLRCONFIG-DELPOLICY2.XML, XSLT, SOLRCONFIG-NATIVELOCK.XML, BAD-SCHEMA-DUP-FIELD.XML, SOLRCONFIG-NOCACHE.XML, SCHEMA-BM25.XML, ADDFIELDS.UPDATEPROCESSOR.JS, SOLRCONFIG-QUERYSENDER-NOQUERY.XML, SOLRCONFIG-ALTDIRECTORY.XML, COMPOUNDDICTIONARY.TXT, SOLRCONFIG-INDEXCONFIG.XML, SOLRCONFIG_PERF.XML, SCHEMA-NOT-REQUIRED-UNIQUE-KEY.XML, BAD-SCHEMA-ANALYZER-CLASS-AND-NESTED.XML, KEEP-2.TXT, BAD-SCHEMA-UNIQUEKEY-IS-COPYFIELD-DEST.XML, SCHEMA12.XML, MAPPING-ISOLATIN1ACCENT.TXT, BAD_SOLRCONFIG.XML, BAD-SCHEMA-EXTERNAL-FILEFIELD.XML]
[junit4:junit4]   2> 3669 T203 oass.SolrIndexSearcher.<init> Opening Searcher@18a76cea main
[junit4:junit4]   2> 3670 T203 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3677 T203 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 3678 T203 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 3679 T203 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 3850 T203 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 4313 T203 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 4314 T203 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4315 T203 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 4316 T203 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4317 T203 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4317 T203 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4318 T203 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4319 T203 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4319 T203 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4320 T203 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 4356 T218 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@18a76cea main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 4372 T203 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 4373 T203 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 4374 T203 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 4478 T203 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 4511 T203 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 4514 T203 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:14355
[junit4:junit4]   2> 4515 T203 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4516 T203 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 4517 T203 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4574 T203 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 4575 T203 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4576 T203 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 4576 T203 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 4577 T203 oasc.CoreContainer.<init> New CoreContainer 275320212
[junit4:junit4]   2> 4578 T203 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4579 T203 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4622 T203 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4660 T203 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 4661 T203 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 4663 T203 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 4665 T203 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4764 T203 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 4858 T203 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 4859 T203 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4933 T203 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 5518 T203 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 5531 T203 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 5535 T203 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5552 T203 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5557 T203 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5562 T203 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5564 T203 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5567 T203 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.handler.component.TermVectorComponentDistributedTest-1351731529440/shard0/data/
[junit4:junit4]   2> 5567 T203 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3b430892
[junit4:junit4]   2> 5569 T203 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.handler.component.TermVectorComponentDistributedTest-1351731529440/shard0/data/index/
[junit4:junit4]   2> 5570 T203 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.handler.component.TermVectorComponentDistributedTest-1351731529440/shard0/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5571 T203 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.handler.component.TermVectorComponentDistributedTest-1351731529440/shard0/data/index forceNew:false
[junit4:junit4]   2> 5574 T203 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c00aff9 lockFactory=org.apache.lucene.store.NativeFSLockFactory@203e4015),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5574 T203 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 5575 T203 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 5578 T203 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 5579 T203 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 5579 T203 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 5580 T203 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 5581 T203 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 5581 T203 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 5582 T203 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 5583 T203 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 5584 T203 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 5585 T203 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 5587 T203 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 5587 T203 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 5589 T203 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 5589 T203 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 5590 T203 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 5591 T203 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 5592 T203 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5593 T203 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5594 T203 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5596 T203 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5597 T203 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5598 T203 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5599 T203 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5601 T203 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 5602 T203 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 5610 T203 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 5614 T203 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 5616 T203 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [THROW.ERROR.ON.ADD.UPDATEPROCESSOR.JS, SOLRCONFIG-HIGHLIGHT.XML, SCHEMA-REQUIRED-FIELDS.XML, BAD-SCHEMA-DUP-DYNAMICFIELD.XML, SCHEMA-MINIMAL.XML, TRIVIAL.UPDATEPROCESSOR1.JS, SCHEMA-REPLICATION2.XML, SOLRCONFIG-CACHING.XML, SOLRCONFIG-REPEATER.XML, SOLRCONFIG-MERGEPOLICY.XML, BAD-SCHEMA-NONTEXT-ANALYZER.XML, CURRENCY.XML, SOLRCONFIG-TLOG.XML, SOLRCONFIG-MASTER.XML, BAD-SCHEMA-UNIQUEKEY-MULTIVALUED.XML, SCHEMA11.XML, SOLRCONFIG-BASIC.XML, DA_COMPOUNDDICTIONARY.TXT, MISSLEADING.EXTENSION.UPDATEPROCESSOR.JS.TXT, SCHEMA-COPYFIELD-TEST.XML, SOLRCONFIG-SLAVE.XML, SOLRCONFIG-PROPINJECT-INDEXDEFAULT.XML, ELEVATE.XML, TRIVIAL.UPDATEPROCESSOR0.JS, BAD-SOLRCONFIG-BOGUS-SCRIPTENGINE-NAME.XML, SCHEMA-CHARFILTERS.XML, SCHEMA-IB.XML, SOLRCONFIG-QUERYSENDER.XML, DA_UTF8.XML, SCHEMA-REPLICATION1.XML, SOLRCONFIG-SNIPPET-PROCESSOR.XML, CONDITIONAL.UPDATEPROCESSOR.JS, MISSING.FUNCTIONS.UPDATEPROCESSOR.JS, HYPHENATION.DTD, SOLRCONFIG-ENABLEPLUGIN.XML, STEMDICT.TXT, SCHEMA-PHRASESUGGEST.XML, HUNSPELL-TEST.AFF, SCHEMA-SNIPPET-TYPE.XML, STOPTYPES-1.TXT, STOPWORDSWRONGENCODING.TXT, SCHEMA-NUMERIC.XML, SOLRCONFIG-TRANSFORMERS.XML, SOLRCONFIG-PROPINJECT.XML, BAD-SCHEMA-NOT-INDEXED-BUT-TF.XML, SOLRCONFIG-SIMPLELOCK.XML, WDFTYPES.TXT, STOPTYPES-2.TXT, SCHEMA-REVERSED.XML, BAD-SCHEMA-CURRENCY-FT-MULTIVALUED.XML, SOLRCONFIG-SPELLCHECKCOMPONENT.XML, SCHEMA-DFR.XML, SOLRCONFIG-PHRASESUGGEST.XML, BAD-SCHEMA-CURRENCY-MULTIVALUED.XML, BAD-SCHEMA-NOT-INDEXED-BUT-POS.XML, KEEP-1.TXT, SCHEMA-SPATIAL.XML, OPEN-EXCHANGE-RATES.JSON, STOPWITHBOM.TXT, SOLRCONFIG-SPELLCHECKER.XML, SCHEMA-BINARYFIELD.XML, SOLRCONFIG-UPDATE-PROCESSOR-CHAINS.XML, BAD-SCHEMA-OMIT-TF-BUT-NOT-POS.XML, BAD-SCHEMA-DUP-FIELDTYPE.XML, SCHEMA-XINCLUDE.XML, SOLRCONFIG-MASTER1.XML, SYNONYMS.TXT, BAD-SCHEMA-CURRENCY-DYNAMIC-MULTIVALUED.XML, SCHEMA.XML, SCHEMA_CODEC.XML, SOLRCONFIG-SOLR-749.XML, SOLRCONFIG-MASTER1-KEEPONEBACKUP.XML, STOP-2.TXT, SOLRCONFIG-FUNCTIONQUERY.XML, SCHEMA-LMDIRICHLET.XML, SOLRCONFIG-TERMINDEX.XML, SOLRCONFIG-ELEVATE.XML, SCHEMA-FOLDING.XML, STOPWORDS.TXT, SCHEMA-STOP-KEEP.XML, BAD-SCHEMA-NOT-INDEXED-BUT-NORMS.XML, SOLRCONFIG-SOLCOREPROPERTIES.XML, STOP-1.TXT, SOLRCONFIG-MASTER2.XML, SCHEMA-SPELLCHECKER.XML, SOLRCONFIG-RESPONSE-LOG-COMPONENT.XML, SOLRCONFIG-LAZYWRITER.XML, SCHEMA-LUCENEMATCHVERSION.XML, FRENCHARTICLES.TXT, BAD-MP-SOLRCONFIG.XML, SCHEMA15.XML, SOLRCONFIG-REQHANDLER.INCL, SCHEMASURROUND.XML, SCHEMA-COLLATEFILTER.XML, SOLRCONFIG-MASTER3.XML, HUNSPELL-TEST.DIC, BAD-SCHEMA-CODEC-GLOBAL-VS-FT-MISMATCH.XML, SOLRCONFIG-XINCLUDE.XML, SOLRCONFIG-SLAVE1.XML, SOLRCONFIG-DELPOLICY1.XML, SCHEMA-SIM.XML, SCHEMA-SNIPPET-FIELD.XML, SCHEMA-COLLATE.XML, STOP-SNOWBALL.TXT, BAD-SCHEMA-SIM-GLOBAL-VS-FT-MISMATCH.XML, PROTWORDS.TXT, SCHEMA-TRIE.XML, SOLRCONFIG_CODEC.XML, BAD-SOLRCONFIG-INVALID-SCRIPTFILE.XML, JASUGGEST.TXT, SCHEMA-TFIDF.XML, SOLRCONFIG-SCRIPT-UPDATEPROCESSOR.XML, SCHEMA-LMJELINEKMERCER.XML, PHRASESUGGEST.TXT, BAD-SOLRCONFIG-MISSING-SCRIPTFILE.XML, SOLRCONFIG-BASIC-LUCENEVERSION31.XML, BAD-SCHEMA-UNIQUEKEY-USES-DEFAULT.XML, OLD_SYNONYMS.TXT, SOLRCONFIG-DELPOLICY2.XML, XSLT, SOLRCONFIG-NATIVELOCK.XML, BAD-SCHEMA-DUP-FIELD.XML, SOLRCONFIG-NOCACHE.XML, SCHEMA-BM25.XML, ADDFIELDS.UPDATEPROCESSOR.JS, SOLRCONFIG-QUERYSENDER-NOQUERY.XML, SOLRCONFIG-ALTDIRECTORY.XML, COMPOUNDDICTIONARY.TXT, SOLRCONFIG-INDEXCONFIG.XML, SOLRCONFIG_PERF.XML, SCHEMA-NOT-REQUIRED-UNIQUE-KEY.XML, BAD-SCHEMA-ANALYZER-CLASS-AND-NESTED.XML, KEEP-2.TXT, BAD-SCHEMA-UNIQUEKEY-IS-COPYFIELD-DEST.XML, SCHEMA12.XML, MAPPING-ISOLATIN1ACCENT.TXT, BAD_SOLRCONFIG.XML, BAD-SCHEMA-EXTERNAL-FILEFIELD.XML]
[junit4:junit4]   2> 5619 T203 oass.SolrIndexSearcher.<init> Opening Searcher@1319b33 main
[junit4:junit4]   2> 5620 T203 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 5621 T203 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 5621 T203 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 5623 T203 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 5641 T203 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 5695 T203 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 5696 T203 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 5696 T203 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 5697 T203 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 5698 T203 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 5698 T203 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 5699 T203 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 5700 T203 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 5700 T203 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 5701 T203 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 5735 T232 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1319b33 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5755 T203 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 5756 T203 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 5756 T203 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 5757 T203 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C0 name=collection1 org.apache.solr.core.SolrCore@31470ee1
[junit4:junit4]   2> 7311 T217 C0 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@153e6d0b lockFactory=org.apache.lucene.store.NativeFSLockFactory@7e768122),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 7315 T217 C0 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 7657 T217 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[0]} 0 681
[junit4:junit4]   2> ASYNC  NEW_CORE C1 name=collection1 org.apache.solr.core.SolrCore@3b1b8618
[junit4:junit4]   2> 7778 T230 C1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c00aff9 lockFactory=org.apache.lucene.store.NativeFSLockFactory@203e4015),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 7779 T230 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 7784 T230 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[0]} 0 9
[junit4:junit4]   2> 7792 T214 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1]} 0 4
[junit4:junit4]   2> 7811 T229 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1]} 0 15
[junit4:junit4]   2> 7818 T217 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[2]} 0 2
[junit4:junit4]   2> 7826 T231 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[2]} 0 4
[junit4:junit4]   2> 7832 T216 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[3]} 0 3
[junit4:junit4]   2> 7837 T230 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[3]} 0 2
[junit4:junit4]   2> 7842 T214 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[4]} 0 2
[junit4:junit4]   2> 7849 T229 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[4]} 0 3
[junit4:junit4]   2> 7854 T217 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[5]} 0 2
[junit4:junit4]   2> 7860 T231 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[5]} 0 2
[junit4:junit4]   2> 7866 T216 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[6]} 0 2
[junit4:junit4]   2> 7871 T230 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[6]} 0 2
[junit4:junit4]   2> 7876 T214 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[7]} 0 1
[junit4:junit4]   2> 7885 T229 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[7]} 0 3
[junit4:junit4]   2> 7891 T217 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[8]} 0 2
[junit4:junit4]   2> 7896 T231 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[8]} 0 2
[junit4:junit4]   2> 8373 T216 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[9]} 0 474
[junit4:junit4]   2> 8389 T231 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[9]} 0 13
[junit4:junit4]   2> 8419 T214 C0 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 8421 T214 C0 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@153e6d0b lockFactory=org.apache.lucene.store.NativeFSLockFactory@7e768122),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@153e6d0b lockFactory=org.apache.lucene.store.NativeFSLockFactory@7e768122),segFN=segments_2,generation=2,filenames=[_0_Lucene41WithOrds_0.tib, _0_Lucene41WithOrds_0.tii, _0_Lucene41WithOrds_0.pos, _0_NestedPulsing_0.tip, _0_NestedPulsing_0.tim, _0_NestedPulsing_0.pos, _0_Pulsing41_0.tim, _0_Pulsing41_0.tip, _0.si, _0_NestedPulsing_0.doc, _0_MockRandom_0.tib, _0.fnm, _0_Lucene41WithOrds_0.doc, _0.tvd, _0.tvf, _0_nrm.cfs, _0_MockRandom_0.skp, _0_MockRandom_0.frq, _0_MockRandom_0.sd, _0_Pulsing41_0.doc, _0_Pulsing41_0.pos, _0.tvx, _0_MockRandom_0.doc, _0_MockRandom_0.pos, _0_nrm.cfe, _0_MockRandom_0.tii, segments_2, _0.fdx, _0_MockRandom_0.pyl, _0.fdt]
[junit4:junit4]   2> 8422 T214 C0 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 8705 T214 C0 oass.SolrIndexSearcher.<init> Opening Searcher@1b7c2ab2 main
[junit4:junit4]   2> 8705 T214 C0 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 8713 T214 C0 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 8725 T218 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b7c2ab2 main{StandardDirectoryReader(segments_2:3 _0(4.1):C10)}
[junit4:junit4]   2> 8727 T214 C0 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 311
[junit4:junit4]   2> 8733 T229 C1 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 8735 T229 C1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c00aff9 lockFactory=org.apache.lucene.store.NativeFSLockFactory@203e4015),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c00aff9 lockFactory=org.apache.lucene.store.NativeFSLockFactory@203e4015),segFN=segments_2,generation=2,filenames=[_0_Lucene41WithOrds_0.tib, _0_NestedPulsing_0.doc, _0_Lucene41WithOrds_0.doc, _0.fnm, _0.tvd, _0_Lucene41WithOrds_0.tii, _0.tvf, _0_Lucene41WithOrds_0.pos, _0_nrm.cfs, _0_NestedPulsing_0.tip, _0_NestedPulsing_0.tim, _0_NestedPulsing_0.pos, _0_Pulsing41_0.pos, _0_Pulsing41_0.doc, _0_MockRandom_0.tip, _0_MockRandom_0.sd, _0.tvx, _0_Pulsing41_0.tim, _0_MockRandom_0.doc, _0_Pulsing41_0.tip, _0_nrm.cfe, _0_MockRandom_0.pos, segments_2, _0.fdx, _0.si, _0_MockRandom_0.tim, _0.fdt]
[junit4:junit4]   2> 8735 T229 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 8746 T229 C1 oass.SolrIndexSearcher.<init> Opening Searcher@4478d5a3 main
[junit4:junit4]   2> 8747 T229 C1 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 8748 T229 C1 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 8755 T232 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4478d5a3 main{StandardDirectoryReader(segments_2:3 _0(4.1):C10)}
[junit4:junit4]   2> 8757 T229 C1 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 24
[junit4:junit4]   2> 9432 T216 C0 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&tv.tf=true&shards.qt=tvrh&q=id:0&tv=true&distrib=false&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=638 
[junit4:junit4]   2> 9649 T229 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&distrib=false&wt=javabin&rows=10&version=2&fl=id&shard.url=[ff01::114]:33332/solr|[ff01::213]:33332/solr|[ff01::083]:33332/solr|127.0.0.1:14355/solr&NOW=1351731538884&tv.tf=true&start=0&q=id:0&tv=true&isShard=true&qt=tvrh&fsv=true} hits=1 status=0 QTime=3 
[junit4:junit4]   2> 9717 T231 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&ids=0&distrib=false&wt=javabin&version=2&shard.url=[ff01::114]:33332/solr|[ff01::213]:33332/solr|[ff01::083]:33332/solr|127.0.0.1:14355/solr&NOW=1351731538884&tv.tf=true&q=id:0&tv=true&qt=tvrh&isShard=true} status=0 QTime=2 
[junit4:junit4]   2> 9727 T230 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards=[ff01::114]:33332/solr|[ff01::213]:33332/solr|[ff01::083]:33332/solr|127.0.0.1:14355/solr&tv.tf=true&shards.qt=tvrh&q=id:0&tv=true&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=273 
[junit4:junit4]   2> 9741 T214 C0 REQ [collection1] webapp=/solr path=/select params={sort=id+asc&fl=*,score&tv.tf=true&shards.qt=tvrh&q=id:0&tv=true&distrib=false&tv.fl=test_basictv,test_offtv&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=3 
[junit4:junit4]   2> 9755 T230 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+asc&shards.qt=tvrh&distrib=false&wt=javabin&rows=10&version=2&fl=id,score&shard.url=127.0.0.1:14355/solr|[ff01::083]:33332/solr|[ff01::213]:33332/solr&NOW=1351731539177&tv.tf=true&start=0&q=id:0&tv=true&tv.fl=test_basictv,test_offtv&qt=tvrh&isShard=true&fsv=true} hits=1 status=0 QTime=2 
[junit4:junit4]   2> 9763 T229 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&ids=0&distrib=false&wt=javabin&version=2&fl=*,score&shard.url=127.0.0.1:14355/solr|[ff01::083]:33332/solr|[ff01::213]:33332/solr&NOW=1351731539177&tv.tf=true&q=id:0&tv=true&tv.fl=test_basictv,test_offtv&qt=tvrh&isShard=true} status=0 QTime=1 
[junit4:junit4]   2> 9765 T231 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+asc&fl=*,score&shards=127.0.0.1:14355/solr|[ff01::083]:33332/solr|[ff01::213]:33332/solr&tv.tf=true&shards.qt=tvrh&q=id:0&tv=true&tv.fl=test_basictv,test_offtv&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=18 
[junit4:junit4]   2> 9772 T233 C0 REQ [collection1] webapp=/solr path=/select params={sort=id+asc&fl=*,score&tv.tf=true&shards.qt=tvrh&q=id:0&tv=true&distrib=false&tv.fl=test_basictv&tv.fl=test_offtv&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 9787 T229 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+asc&shards.qt=tvrh&distrib=false&wt=javabin&rows=10&version=2&fl=id,score&shard.url=[ff01::213]:33332/solr|[ff01::114]:33332/solr|127.0.0.1:14355/solr&NOW=1351731539207&tv.tf=true&start=0&q=id:0&tv=true&tv.fl=test_basictv&tv.fl=test_offtv&qt=tvrh&isShard=true&fsv=true} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 9793 T231 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&ids=0&distrib=false&wt=javabin&version=2&fl=*,score&shard.url=[ff01::213]:33332/solr|[ff01::114]:33332/solr|127.0.0.1:14355/solr&NOW=1351731539207&tv.tf=true&q=id:0&tv=true&tv.fl=test_basictv&tv.fl=test_offtv&qt=tvrh&isShard=true} status=0 QTime=1 
[junit4:junit4]   2> 9795 T230 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+asc&fl=*,score&shards=[ff01::213]:33332/solr|[ff01::114]:33332/solr|127.0.0.1:14355/solr&tv.tf=true&shards.qt=tvrh&q=id:0&tv=true&tv.fl=test_basictv&tv.fl=test_offtv&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=18 
[junit4:junit4]   2> 9802 T233 C0 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&fl=*,score&tv.tf=true&shards.qt=tvrh&q=id:0&tv=true&distrib=false&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 9814 T231 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&distrib=false&wt=javabin&rows=10&version=2&fl=id,score&shard.url=127.0.0.1:14355/solr&NOW=1351731539238&tv.tf=true&start=0&q=id:0&tv=true&qt=tvrh&isShard=true&fsv=true} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 9821 T230 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&ids=0&distrib=false&wt=javabin&version=2&fl=*,score&shard.url=127.0.0.1:14355/solr&NOW=1351731539238&tv.tf=true&q=id:0&tv=true&qt=tvrh&isShard=true} status=0 QTime=1 
[junit4:junit4]   2> 9824 T229 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&fl=*,score&shards=127.0.0.1:14355/solr&tv.tf=true&shards.qt=tvrh&q=id:0&tv=true&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=16 
[junit4:junit4]   2> 9881 T217 C0 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&fl=score,test_basictv,[docid],test_postv,val:sum(3,4)&tv.tf=true&shards.qt=tvrh&q=id:0&tv=true&distrib=false&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=52 
[junit4:junit4]   2> 9948 T230 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&distrib=false&wt=javabin&rows=10&version=2&fl=id,score&shard.url=127.0.0.1:14355/solr|[ff01::213]:33332/solr&NOW=1351731539372&tv.tf=true&start=0&q=id:0&tv=true&qt=tvrh&isShard=true&fsv=true} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 9956 T229 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&ids=0&distrib=false&wt=javabin&version=2&fl=score,test_basictv,[docid],test_postv,val:sum(3,4)&fl=id&shard.url=127.0.0.1:14355/solr|[ff01::213]:33332/solr&NOW=1351731539372&tv.tf=true&q=id:0&tv=true&qt=tvrh&isShard=true} status=0 QTime=2 
[junit4:junit4]   2> 9957 T231 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&fl=score,test_basictv,[docid],test_postv,val:sum(3,4)&shards=127.0.0.1:14355/solr|[ff01::213]:33332/solr&tv.tf=true&shards.qt=tvrh&q=id:0&tv=true&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=15 
[junit4:junit4]   2> 9964 T216 C0 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&fl=score,test_basictv&fl=[docid],test_postv,val:sum(3,4)&tv.tf=true&shards.qt=tvrh&q=id:0&tv=true&distrib=false&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 9975 T229 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&distrib=false&wt=javabin&rows=10&version=2&fl=id,score&shard.url=[ff01::114]:33332/solr|[ff01::083]:33332/solr|[ff01::213]:33332/solr|127.0.0.1:14355/solr&NOW=1351731539399&tv.tf=true&start=0&q=id:0&tv=true&qt=tvrh&isShard=true&fsv=true} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 9983 T231 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&ids=0&distrib=false&wt=javabin&version=2&fl=score,test_basictv&fl=[docid],test_postv,val:sum(3,4)&fl=id&shard.url=[ff01::114]:33332/solr|[ff01::083]:33332/solr|[ff01::213]:33332/solr|127.0.0.1:14355/solr&NOW=1351731539399&tv.tf=true&q=id:0&tv=true&qt=tvrh&isShard=true} status=0 QTime=1 
[junit4:junit4]   2> 9984 T230 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&fl=score,test_basictv&fl=[docid],test_postv,val:sum(3,4)&shards=[ff01::114]:33332/solr|[ff01::083]:33332/solr|[ff01::213]:33332/solr|127.0.0.1:14355/solr&tv.tf=true&shards.qt=tvrh&q=id:0&tv=true&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=15 
[junit4:junit4]   2> 10001 T214 C0 REQ [collection1] webapp=/solr path=/select params={sort=id+asc&shards.qt=tvrh&tv.offsets=true&tv.positions=true&distrib=false&tv.df=true&wt=javabin&version=2&tv.tf=true&q=id:0&tv=true&tv.tf_idf=true&qt=tvrh} hits=1 status=0 QTime=11 
[junit4:junit4]   2> 10012 T231 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+asc&shards.qt=tvrh&tv.offsets=true&tv.positions=true&distrib=false&tv.df=true&wt=javabin&rows=10&version=2&fl=id&shard.url=127.0.0.1:14355/solr&NOW=1351731539436&tv.tf=true&start=0&q=id:0&tv=true&tv.tf_idf=true&qt=tvrh&isShard=true&fsv=true} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 10019 T230 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&ids=0&tv.offsets=true&tv.positions=true&distrib=false&tv.df=true&wt=javabin&version=2&shard.url=127.0.0.1:14355/solr&NOW=1351731539436&tv.tf=true&q=id:0&tv=true&tv.tf_idf=true&qt=tvrh&isShard=true} status=0 QTime=2 
[junit4:junit4]   2> 10021 T229 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+asc&shards.qt=tvrh&tv.offsets=true&tv.positions=true&tv.df=true&wt=javabin&version=2&shards=127.0.0.1:14355/solr&tv.tf=true&q=id:0&tv=true&tv.tf_idf=true&qt=tvrh} hits=1 status=0 QTime=15 
[junit4:junit4]   2> 10030 T233 C0 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&q=id:0&tv=true&tv.all=true&distrib=false&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=2 
[junit4:junit4]   2> 10040 T230 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&tv.all=true&distrib=false&wt=javabin&rows=10&version=2&fl=id&shard.url=[ff01::114]:33332/solr|127.0.0.1:14355/solr&NOW=1351731539465&start=0&q=id:0&tv=true&qt=tvrh&isShard=true&fsv=true} hits=1 status=0 QTime=0 
[junit4:junit4]   2> 10048 T229 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&ids=0&tv.all=true&distrib=false&wt=javabin&version=2&shard.url=[ff01::114]:33332/solr|127.0.0.1:14355/solr&NOW=1351731539465&q=id:0&tv=true&qt=tvrh&isShard=true} status=0 QTime=2 
[junit4:junit4]   2> 10051 T231 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards=[ff01::114]:33332/solr|127.0.0.1:14355/solr&shards.qt=tvrh&q=id:0&tv=true&tv.all=true&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=16 
[junit4:junit4]   2> 10058 T217 C0 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&f.test_basictv.tv.df=false&tv.offsets=true&tv.positions=true&distrib=false&tv.df=true&f.test_basictv.tv.tf_idf=false&wt=javabin&version=2&tv.tf=true&f.test_basictv.tv.tf=false&q=id:0&tv=true&f.test_posofftv.tv.positions=false&tv.fl=test_basictv,test_notv,test_postv,test_offtv,test_posofftv&tv.tf_idf=true&qt=tvrh&f.test_offtv.tv.offsets=false} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 10074 T229 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&distrib=false&tv.positions=true&f.test_basictv.tv.tf_idf=false&version=2&NOW=1351731539494&shard.url=[ff01::114]:33332/solr|[ff01::083]:33332/solr|[ff01::213]:33332/solr|127.0.0.1:14355/solr&fl=id&f.test_posofftv.tv.positions=false&qt=tvrh&fsv=true&f.test_basictv.tv.df=false&tv.offsets=true&tv.df=true&wt=javabin&rows=10&tv.tf=true&start=0&f.test_basictv.tv.tf=false&q=id:0&tv=true&tv.tf_idf=true&tv.fl=test_basictv,test_notv,test_postv,test_offtv,test_posofftv&isShard=true&f.test_offtv.tv.offsets=false} hits=1 status=0 QTime=3 
[junit4:junit4]   2> 10083 T231 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&tv.positions=true&distrib=false&f.test_basictv.tv.tf_idf=false&version=2&shard.url=[ff01::114]:33332/solr|[ff01::083]:33332/solr|[ff01::213]:33332/solr|127.0.0.1:14355/solr&NOW=1351731539494&f.test_posofftv.tv.positions=false&qt=tvrh&f.test_basictv.tv.df=false&ids=0&tv.offsets=true&tv.df=true&wt=javabin&tv.tf=true&f.test_basictv.tv.tf=false&q=id:0&tv=true&tv.fl=test_basictv,test_notv,test_postv,test_offtv,test_posofftv&tv.tf_idf=true&isShard=true&f.test_offtv.tv.offsets=false} status=0 QTime=2 
[junit4:junit4]   2> 10086 T230 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&f.test_basictv.tv.df=false&tv.offsets=true&tv.positions=true&tv.df=true&f.test_basictv.tv.tf_idf=false&wt=javabin&version=2&shards=[ff01::114]:33332/solr|[ff01::083]:33332/solr|[ff01::213]:33332/solr|127.0.0.1:14355/solr&tv.tf=true&f.test_basictv.tv.tf=false&q=id:0&tv=true&f.test_posofftv.tv.positions=false&tv.fl=test_basictv,test_notv,test_postv,test_offtv,test_posofftv&tv.tf_idf=true&qt=tvrh&f.test_offtv.tv.offsets=false} hits=1 status=0 QTime=22 
[junit4:junit4]   2> 10092 T216 C0 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&tv.tf=true&shards.qt=tvrh&q=id:7&tv=true&distrib=false&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 10102 T231 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&distrib=false&wt=javabin&rows=10&version=2&fl=id&shard.url=[ff01::213]:33332/solr|127.0.0.1:14355/solr|[ff01::114]:33332/solr|[ff01::083]:33332/solr&NOW=1351731539527&tv.tf=true&start=0&q=id:7&tv=true&isShard=true&qt=tvrh&fsv=true} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 10110 T230 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&ids=7&distrib=false&wt=javabin&version=2&shard.url=[ff01::213]:33332/solr|127.0.0.1:14355/solr|[ff01::114]:33332/solr|[ff01::083]:33332/solr&NOW=1351731539527&tv.tf=true&q=id:7&tv=true&isShard=true&qt=tvrh} status=0 QTime=1 
[junit4:junit4]   2> 10112 T229 C1 REQ [collection1] webapp=/solr path=/select params={shards=[ff01::213]:33332/solr|127.0.0.1:14355/solr|[ff01::114]:33332/solr|[ff01::083]:33332/solr&sort=id+desc&tv.tf=true&shards.qt=tvrh&q=id:7&tv=true&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=15 
[junit4:junit4]   2> 10119 T214 C0 REQ [collection1] webapp=/solr path=/select params={fl=*,score&sort=id+asc&tv.tf=true&shards.qt=tvrh&q=id:7&tv=true&distrib=false&tv.fl=test_basictv,test_offtv&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 10130 T230 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+asc&shards.qt=tvrh&distrib=false&wt=javabin&rows=10&version=2&fl=id,score&shard.url=[ff01::213]:33332/solr|127.0.0.1:14355/solr|[ff01::114]:33332/solr&NOW=1351731539554&tv.tf=true&start=0&q=id:7&tv=true&tv.fl=test_basictv,test_offtv&isShard=true&qt=tvrh&fsv=true} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 10137 T229 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&ids=7&distrib=false&wt=javabin&version=2&fl=*,score&shard.url=[ff01::213]:33332/solr|127.0.0.1:14355/solr|[ff01::114]:33332/solr&NOW=1351731539554&tv.tf=true&q=id:7&tv=true&tv.fl=test_basictv,test_offtv&isShard=true&qt=tvrh} status=0 QTime=1 
[junit4:junit4]   2> 10140 T231 C1 REQ [collection1] webapp=/solr path=/select params={fl=*,score&shards=[ff01::213]:33332/solr|127.0.0.1:14355/solr|[ff01::114]:33332/solr&sort=id+asc&tv.tf=true&shards.qt=tvrh&q=id:7&tv=true&tv.fl=test_basictv,test_offtv&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=16 
[junit4:junit4]   2> 10146 T233 C0 REQ [collection1] webapp=/solr path=/select params={fl=*,score&sort=id+asc&tv.tf=true&shards.qt=tvrh&q=id:7&tv=true&distrib=false&tv.fl=test_basictv&tv.fl=test_offtv&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 10158 T231 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+asc&shards.qt=tvrh&distrib=false&wt=javabin&rows=10&version=2&fl=id,score&shard.url=127.0.0.1:14355/solr&NOW=1351731539582&tv.tf=true&start=0&q=id:7&tv=true&tv.fl=test_basictv&tv.fl=test_offtv&isShard=true&qt=tvrh&fsv=true} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 10164 T230 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&ids=7&distrib=false&wt=javabin&version=2&fl=*,score&shard.url=127.0.0.1:14355/solr&NOW=1351731539582&tv.tf=true&q=id:7&tv=true&tv.fl=test_basictv&tv.fl=test_offtv&isShard=true&qt=tvrh} status=0 QTime=1 
[junit4:junit4]   2> 10166 T229 C1 REQ [collection1] webapp=/solr path=/select params={fl=*,score&shards=127.0.0.1:14355/solr&sort=id+asc&tv.tf=true&shards.qt=tvrh&q=id:7&tv=true&tv.fl=test_basictv&tv.fl=test_offtv&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=14 
[junit4:junit4]   2> 10173 T216 C0 REQ [collection1] webapp=/solr path=/select params={fl=*,score&sort=id+desc&tv.tf=true&shards.qt=tvrh&q=id:7&tv=true&distrib=false&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 10184 T230 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&distrib=false&wt=javabin&rows=10&version=2&fl=id,score&shard.url=[ff01::083]:33332/solr|127.0.0.1:14355/solr|[ff01::114]:33332/solr&NOW=1351731539608&tv.tf=true&start=0&q=id:7&tv=true&isShard=true&qt=tvrh&fsv=true} hits=1 status=0 QTime=2 
[junit4:junit4]   2> 10191 T229 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&ids=7&distrib=false&wt=javabin&version=2&fl=*,score&shard.url=[ff01::083]:33332/solr|127.0.0.1:14355/solr|[ff01::114]:33332/solr&NOW=1351731539608&tv.tf=true&q=id:7&tv=true&isShard=true&qt=tvrh} status=0 QTime=1 
[junit4:junit4]   2> 10193 T231 C1 REQ [collection1] webapp=/solr path=/select params={fl=*,score&shards=[ff01::083]:33332/solr|127.0.0.1:14355/solr|[ff01::114]:33332/solr&sort=id+desc&tv.tf=true&shards.qt=tvrh&q=id:7&tv=true&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=15 
[junit4:junit4]   2> 10200 T216 C0 REQ [collection1] webapp=/solr path=/select params={fl=score,test_basictv,[docid],test_postv,val:sum(3,4)&sort=id+desc&tv.tf=true&shards.qt=tvrh&q=id:7&tv=true&distrib=false&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 10211 T229 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&distrib=false&wt=javabin&rows=10&version=2&fl=id,score&shard.url=[ff01::083]:33332/solr|127.0.0.1:14355/solr|[ff01::213]:33332/solr&NOW=1351731539635&tv.tf=true&start=0&q=id:7&tv=true&isShard=true&qt=tvrh&fsv=true} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 10217 T231 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&ids=7&distrib=false&wt=javabin&version=2&fl=score,test_basictv,[docid],test_postv,val:sum(3,4)&fl=id&shard.url=[ff01::083]:33332/solr|127.0.0.1:14355/solr|[ff01::213]:33332/solr&NOW=1351731539635&tv.tf=true&q=id:7&tv=true&isShard=true&qt=tvrh} status=0 QTime=1 
[junit4:junit4]   2> 10219 T230 C1 REQ [collection1] webapp=/solr path=/select params={fl=score,test_basictv,[docid],test_postv,val:sum(3,4)&shards=[ff01::083]:33332/solr|127.0.0.1:14355/solr|[ff01::213]:33332/solr&sort=id+desc&tv.tf=true&shards.qt=tvrh&q=id:7&tv=true&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=14 
[junit4:junit4]   2> 10225 T233 C0 REQ [collection1] webapp=/solr path=/select params={fl=score,test_basictv&fl=[docid],test_postv,val:sum(3,4)&sort=id+desc&tv.tf=true&shards.qt=tvrh&q=id:7&tv=true&distrib=false&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 10235 T231 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&distrib=false&wt=javabin&rows=10&version=2&fl=id,score&shard.url=[ff01::083]:33332/solr|127.0.0.1:14355/solr&NOW=1351731539660&tv.tf=true&start=0&q=id:7&tv=true&isShard=true&qt=tvrh&fsv=true} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 10242 T230 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&ids=7&distrib=false&wt=javabin&version=2&fl=score,test_basictv&fl=[docid],test_postv,val:sum(3,4)&fl=id&shard.url=[ff01::083]:33332/solr|127.0.0.1:14355/solr&NOW=1351731539660&tv.tf=true&q=id:7&tv=true&isShard=true&qt=tvrh} status=0 QTime=1 
[junit4:junit4]   2> 10243 T229 C1 REQ [collection1] webapp=/solr path=/select params={fl=score,test_basictv&fl=[docid],test_postv,val:sum(3,4)&shards=[ff01::083]:33332/solr|127.0.0.1:14355/solr&sort=id+desc&tv.tf=true&shards.qt=tvrh&q=id:7&tv=true&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=13 
[junit4:junit4]   2> 10250 T217 C0 REQ [collection1] webapp=/solr path=/select params={sort=id+asc&shards.qt=tvrh&tv.offsets=true&tv.positions=true&distrib=false&tv.df=true&wt=javabin&version=2&tv.tf=true&q=id:7&tv=true&tv.tf_idf=true&qt=tvrh} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 10260 T230 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+asc&shards.qt=tvrh&tv.offsets=true&tv.positions=true&distrib=false&tv.df=true&wt=javabin&rows=10&version=2&fl=id&shard.url=[ff01::083]:33332/solr|[ff01::213]:33332/solr|127.0.0.1:14355/solr|[ff01::114]:33332/solr&NOW=1351731539685&tv.tf=true&start=0&q=id:7&tv=true&tv.tf_idf=true&isShard=true&qt=tvrh&fsv=true} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 10268 T229 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&ids=7&tv.offsets=true&tv.positions=true&distrib=false&tv.df=true&wt=javabin&version=2&shard.url=[ff01::083]:33332/solr|[ff01::213]:33332/solr|127.0.0.1:14355/solr|[ff01::114]:33332/solr&NOW=1351731539685&tv.tf=true&q=id:7&tv=true&tv.tf_idf=true&isShard=true&qt=tvrh} status=0 QTime=2 
[junit4:junit4]   2> 10270 T231 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+asc&shards.qt=tvrh&tv.offsets=true&tv.positions=true&tv.df=true&wt=javabin&version=2&shards=[ff01::083]:33332/solr|[ff01::213]:33332/solr|127.0.0.1:14355/solr|[ff01::114]:33332/solr&tv.tf=true&q=id:7&tv=true&tv.tf_idf=true&qt=tvrh} hits=1 status=0 QTime=15 
[junit4:junit4]   2> 10276 T214 C0 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&q=id:7&tv=true&tv.all=true&distrib=false&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 10286 T229 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&tv.all=true&distrib=false&wt=javabin&rows=10&version=2&fl=id&shard.url=127.0.0.1:14355/solr&NOW=1351731539711&start=0&q=id:7&tv=true&isShard=true&qt=tvrh&fsv=true} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 10293 T231 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&ids=7&tv.all=true&distrib=false&wt=javabin&version=2&shard.url=127.0.0.1:14355/solr&NOW=1351731539711&q=id:7&tv=true&isShard=true&qt=tvrh} status=0 QTime=1 
[junit4:junit4]   2> 10295 T230 C1 REQ [collection1] webapp=/solr path=/select params={shards=127.0.0.1:14355/solr&sort=id+desc&shards.qt=tvrh&q=id:7&tv=true&tv.all=true&qt=tvrh&wt=javabin&version=2} hits=1 status=0 QTime=14 
[junit4:junit4]   2> 10302 T216 C0 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&f.test_basictv.tv.df=false&tv.offsets=true&tv.positions=true&distrib=false&tv.df=true&f.test_basictv.tv.tf_idf=false&wt=javabin&version=2&tv.tf=true&f.test_basictv.tv.tf=false&q=id:7&tv=true&f.test_posofftv.tv.positions=false&tv.tf_idf=true&tv.fl=test_basictv,test_notv,test_postv,test_offtv,test_posofftv&qt=tvrh&f.test_offtv.tv.offsets=false} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 10314 T231 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&tv.positions=true&distrib=false&f.test_basictv.tv.tf_idf=false&version=2&fl=id&shard.url=127.0.0.1:14355/solr|[ff01::114]:33332/solr|[ff01::083]:33332/solr&NOW=1351731539737&f.test_posofftv.tv.positions=false&qt=tvrh&fsv=true&f.test_basictv.tv.df=false&tv.offsets=true&tv.df=true&wt=javabin&rows=10&tv.tf=true&f.test_basictv.tv.tf=false&start=0&q=id:7&tv=true&tv.tf_idf=true&tv.fl=test_basictv,test_notv,test_postv,test_offtv,test_posofftv&isShard=true&f.test_offtv.tv.offsets=false} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 10323 T230 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&tv.positions=true&distrib=false&f.test_basictv.tv.tf_idf=false&version=2&shard.url=127.0.0.1:14355/solr|[ff01::114]:33332/solr|[ff01::083]:33332/solr&NOW=1351731539737&f.test_posofftv.tv.positions=false&qt=tvrh&f.test_basictv.tv.df=false&ids=7&tv.offsets=true&tv.df=true&wt=javabin&tv.tf=true&f.test_basictv.tv.tf=false&q=id:7&tv=true&tv.tf_idf=true&tv.fl=test_basictv,test_notv,test_postv,test_offtv,test_posofftv&isShard=true&f.test_offtv.tv.offsets=false} status=0 QTime=2 
[junit4:junit4]   2> 10327 T229 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&f.test_basictv.tv.df=false&tv.offsets=true&tv.positions=true&tv.df=true&f.test_basictv.tv.tf_idf=false&wt=javabin&version=2&shards=127.0.0.1:14355/solr|[ff01::114]:33332/solr|[ff01::083]:33332/solr&tv.tf=true&f.test_basictv.tv.tf=false&q=id:7&tv=true&f.test_posofftv.tv.positions=false&tv.tf_idf=true&tv.fl=test_basictv,test_notv,test_postv,test_offtv,test_posofftv&qt=tvrh&f.test_offtv.tv.offsets=false} hits=1 status=0 QTime=20 
[junit4:junit4]   2> 10406 T233 C0 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&tv.tf=true&shards.qt=tvrh&q=id:[3+TO+6]&tv=true&distrib=false&qt=tvrh&wt=javabin&version=2} hits=4 status=0 QTime=72 
[junit4:junit4]   2> 10418 T230 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&distrib=false&wt=javabin&rows=10&version=2&fl=id&shard.url=127.0.0.1:14355/solr|[ff01::213]:33332/solr|[ff01::114]:33332/solr&NOW=1351731539843&tv.tf=true&start=0&q=id:[3+TO+6]&tv=true&isShard=true&qt=tvrh&fsv=true} hits=4 status=0 QTime=1 
[junit4:junit4]   2> 10427 T229 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&ids=3,6,5,4&distrib=false&wt=javabin&version=2&shard.url=127.0.0.1:14355/solr|[ff01::213]:33332/solr|[ff01::114]:33332/solr&NOW=1351731539843&tv.tf=true&q=id:[3+TO+6]&tv=true&isShard=true&qt=tvrh} status=0 QTime=3 
[junit4:junit4]   2> 10432 T231 C1 REQ [collection1] webapp=/solr path=/select params={shards=127.0.0.1:14355/solr|[ff01::213]:33332/solr|[ff01::114]:33332/solr&sort=id+desc&tv.tf=true&shards.qt=tvrh&q=id:[3+TO+6]&tv=true&qt=tvrh&wt=javabin&version=2} hits=4 status=0 QTime=19 
[junit4:junit4]   2> 10439 T217 C0 REQ [collection1] webapp=/solr path=/select params={fl=*,score&sort=id+asc&tv.tf=true&shards.qt=tvrh&q=id:[3+TO+6]&tv=true&distrib=false&tv.fl=test_basictv,test_offtv&qt=tvrh&wt=javabin&version=2} hits=4 status=0 QTime=1 
[junit4:junit4]   2> 10451 T229 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+asc&shards.qt=tvrh&distrib=false&wt=javabin&rows=10&version=2&fl=id,score&shard.url=127.0.0.1:14355/solr|[ff01::213]:33332/solr&NOW=1351731539875&tv.tf=true&start=0&q=id:[3+TO+6]&tv=true&tv.fl=test_basictv,test_offtv&isShard=true&qt=tvrh&fsv=true} hits=4 status=0 QTime=2 
[junit4:junit4]   2> 10458 T231 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&ids=3,6,5,4&distrib=false&wt=javabin&version=2&fl=*,score&shard.url=127.0.0.1:14355/solr|[ff01::213]:33332/solr&NOW=1351731539875&tv.tf=true&q=id:[3+TO+6]&tv=true&tv.fl=test_basictv,test_offtv&isShard=true&qt=tvrh} status=0 QTime=1 
[junit4:junit4]   2> 10462 T230 C1 REQ [collection1] webapp=/solr path=/select params={fl=*,score&shards=127.0.0.1:14355/solr|[ff01::213]:33332/solr&sort=id+asc&tv.tf=true&shards.qt=tvrh&q=id:[3+TO+6]&tv=true&tv.fl=test_basictv,test_offtv&qt=tvrh&wt=javabin&version=2} hits=4 status=0 QTime=17 
[junit4:junit4]   2> 10469 T214 C0 REQ [collection1] webapp=/solr path=/select params={fl=*,score&sort=id+asc&tv.tf=true&shards.qt=tvrh&q=id:[3+TO+6]&tv=true&distrib=false&tv.fl=test_basictv&tv.fl=test_offtv&qt=tvrh&wt=javabin&version=2} hits=4 status=0 QTime=1 
[junit4:junit4]   2> 10481 T231 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+asc&shards.qt=tvrh&distrib=false&wt=javabin&rows=10&version=2&fl=id,score&shard.url=[ff01::083]:33332/solr|[ff01::114]:33332/solr|127.0.0.1:14355/solr|[ff01::213]:33332/solr&NOW=1351731539906&tv.tf=true&start=0&q=id:[3+TO+6]&tv=true&tv.fl=test_basictv&tv.fl=test_offtv&isShard=true&qt=tvrh&fsv=true} hits=4 status=0 QTime=1 
[junit4:junit4]   2> 10488 T230 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&ids=3,6,5,4&distrib=false&wt=javabin&version=2&fl=*,score&shard.url=[ff01::083]:33332/solr|[ff01::114]:33332/solr|127.0.0.1:14355/solr|[ff01::213]:33332/solr&NOW=1351731539906&tv.tf=true&q=id:[3+TO+6]&tv=true&tv.fl=test_basictv&tv.fl=test_offtv&isShard=true&qt=tvrh} status=0 QTime=1 
[junit4:junit4]   2> 10492 T229 C1 REQ [collection1] webapp=/solr path=/select params={fl=*,score&shards=[ff01::083]:33332/solr|[ff01::114]:33332/solr|127.0.0.1:14355/solr|[ff01::213]:33332/solr&sort=id+asc&tv.tf=true&shards.qt=tvrh&q=id:[3+TO+6]&tv=true&tv.fl=test_basictv&tv.fl=test_offtv&qt=tvrh&wt=javabin&version=2} hits=4 status=0 QTime=16 
[junit4:junit4]   2> 10501 T216 C0 REQ [collection1] webapp=/solr path=/select params={fl=*,score&sort=id+desc&tv.tf=true&shards.qt=tvrh&q=id:[3+TO+6]&tv=true&distrib=false&qt=tvrh&wt=javabin&version=2} hits=4 status=0 QTime=1 
[junit4:junit4]   2> 10512 T230 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&distrib=false&wt=javabin&rows=10&version=2&fl=id,score&shard.url=[ff01::213]:33332/solr|[ff01::114]:33332/solr|[ff01::083]:33332/solr|127.0.0.1:14355/solr&NOW=1351731539937&tv.tf=true&start=0&q=id:[3+TO+6]&tv=true&isShard=true&qt=tvrh&fsv=true} hits=4 status=0 QTime=1 
[junit4:junit4]   2> 10520 T229 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&ids=3,6,5,4&distrib=false&wt=javabin&version=2&fl=*,score&shard.url=[ff01::213]:33332/solr|[ff01::114]:33332/solr|[ff01::083]:33332/solr|127.0.0.1:14355/solr&NOW=1351731539937&tv.tf=true&q=id:[3+TO+6]&tv=true&isShard=true&qt=tvrh} status=0 QTime=2 
[junit4:junit4]   2> 10525 T231 C1 REQ [collection1] webapp=/solr path=/select params={fl=*,score&shards=[ff01::213]:33332/solr|[ff01::114]:33332/solr|[ff01::083]:33332/solr|127.0.0.1:14355/solr&sort=id+desc&tv.tf=true&shards.qt=tvrh&q=id:[3+TO+6]&tv=true&qt=tvrh&wt=javabin&version=2} hits=4 status=0 QTime=18 
[junit4:junit4]   2> 10533 T233 C0 REQ [collection1] webapp=/solr path=/select params={fl=score,test_basictv,[docid],test_postv,val:sum(3,4)&sort=id+desc&tv.tf=true&shards.qt=tvrh&q=id:[3+TO+6]&tv=true&distrib=false&qt=tvrh&wt=javabin&version=2} hits=4 status=0 QTime=1 
[junit4:junit4]   2> 10543 T229 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&distrib=false&wt=javabin&rows=10&version=2&fl=id,score&shard.url=127.0.0.1:14355/solr|[ff01::213]:33332/solr|[ff01::114]:33332/solr|[ff01::083]:33332/solr&NOW=1351731539967&tv.tf=true&start=0&q=id:[3+TO+6]&tv=true&isShard=true&qt=tvrh&fsv=true} hits=4 status=0 QTime=1 
[junit4:junit4]   2> 10549 T230 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&ids=3,6,5,4&distrib=false&wt=javabin&version=2&fl=score,test_basictv,[docid],test_postv,val:sum(3,4)&fl=id&shard.url=127.0.0.1:14355/solr|[ff01::213]:33332/solr|[ff01::114]:33332/solr|[ff01::083]:33332/solr&NOW=1351731539967&tv.tf=true&q=id:[3+TO+6]&tv=true&isShard=true&qt=tvrh} status=0 QTime=1 
[junit4:junit4]   2> 10551 T231 C1 REQ [collection1] webapp=/solr path=/select params={fl=score,test_basictv,[docid],test_postv,val:sum(3,4)&shards=127.0.0.1:14355/solr|[ff01::213]:33332/solr|[ff01::114]:33332/solr|[ff01::083]:33332/solr&sort=id+desc&tv.tf=true&shards.qt=tvrh&q=id:[3+TO+6]&tv=true&qt=tvrh&wt=javabin&version=2} hits=4 status=0 QTime=14 
[junit4:junit4]   2> 10556 T217 C0 REQ [collection1] webapp=/solr path=/select params={fl=score,test_basictv&fl=[docid],test_postv,val:sum(3,4)&sort=id+desc&tv.tf=true&shards.qt=tvrh&q=id:[3+TO+6]&tv=true&distrib=false&qt=tvrh&wt=javabin&version=2} hits=4 status=0 QTime=1 
[junit4:junit4]   2> 10565 T231 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&distrib=false&wt=javabin&rows=10&version=2&fl=id,score&shard.url=[ff01::213]:33332/solr|[ff01::083]:33332/solr|127.0.0.1:14355/solr&NOW=1351731539990&tv.tf=true&start=0&q=id:[3+TO+6]&tv=true&isShard=true&qt=tvrh&fsv=true} hits=4 status=0 QTime=1 
[junit4:junit4]   2> 10571 T229 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&ids=3,6,5,4&distrib=false&wt=javabin&version=2&fl=score,test_basictv&fl=[docid],test_postv,val:sum(3,4)&fl=id&shard.url=[ff01::213]:33332/solr|[ff01::083]:33332/solr|127.0.0.1:14355/solr&NOW=1351731539990&tv.tf=true&q=id:[3+TO+6]&tv=true&isShard=true&qt=tvrh} status=0 QTime=1 
[junit4:junit4]   2> 10574 T230 C1 REQ [collection1] webapp=/solr path=/select params={fl=score,test_basictv&fl=[docid],test_postv,val:sum(3,4)&shards=[ff01::213]:33332/solr|[ff01::083]:33332/solr|127.0.0.1:14355/solr&sort=id+desc&tv.tf=true&shards.qt=tvrh&q=id:[3+TO+6]&tv=true&qt=tvrh&wt=javabin&version=2} hits=4 status=0 QTime=13 
[junit4:junit4]   2> 10580 T216 C0 REQ [collection1] webapp=/solr path=/select params={sort=id+asc&shards.qt=tvrh&tv.offsets=true&tv.positions=true&distrib=false&tv.df=true&wt=javabin&version=2&tv.tf=true&q=id:[3+TO+6]&tv=true&tv.tf_idf=true&qt=tvrh} hits=4 status=0 QTime=2 
[junit4:junit4]   2> 10590 T229 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+asc&shards.qt=tvrh&tv.offsets=true&tv.positions=true&distrib=false&tv.df=true&wt=javabin&rows=10&version=2&fl=id&shard.url=127.0.0.1:14355/solr|[ff01::083]:33332/solr|[ff01::213]:33332/solr|[ff01::114]:33332/solr&NOW=1351731540015&tv.tf=true&start=0&q=id:[3+TO+6]&tv=true&tv.tf_idf=true&isShard=true&qt=tvrh&fsv=true} hits=4 status=0 QTime=0 
[junit4:junit4]   2> 10598 T230 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&ids=3,6,5,4&tv.offsets=true&tv.positions=true&distrib=false&tv.df=true&wt=javabin&version=2&shard.url=127.0.0.1:14355/solr|[ff01::083]:33332/solr|[ff01::213]:33332/solr|[ff01::114]:33332/solr&NOW=1351731540015&tv.tf=true&q=id:[3+TO+6]&tv=true&tv.tf_idf=true&isShard=true&qt=tvrh} status=0 QTime=3 
[junit4:junit4]   2> 10601 T231 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+asc&shards.qt=tvrh&tv.offsets=true&tv.positions=true&tv.df=true&wt=javabin&version=2&shards=127.0.0.1:14355/solr|[ff01::083]:33332/solr|[ff01::213]:33332/solr|[ff01::114]:33332/solr&tv.tf=true&q=id:[3+TO+6]&tv=true&tv.tf_idf=true&qt=tvrh} hits=4 status=0 QTime=16 
[junit4:junit4]   2> 10609 T214 C0 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&q=id:[3+TO+6]&tv=true&tv.all=true&distrib=false&qt=tvrh&wt=javabin&version=2} hits=4 status=0 QTime=3 
[junit4:junit4]   2> 10618 T230 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&tv.all=true&distrib=false&wt=javabin&rows=10&version=2&fl=id&shard.url=127.0.0.1:14355/solr|[ff01::213]:33332/solr&NOW=1351731540044&start=0&q=id:[3+TO+6]&tv=true&isShard=true&qt=tvrh&fsv=true} hits=4 status=0 QTime=0 
[junit4:junit4]   2> 10626 T231 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&ids=3,6,5,4&tv.all=true&distrib=false&wt=javabin&version=2&shard.url=127.0.0.1:14355/solr|[ff01::213]:33332/solr&NOW=1351731540044&q=id:[3+TO+6]&tv=true&isShard=true&qt=tvrh} status=0 QTime=3 
[junit4:junit4]   2> 10628 T229 C1 REQ [collection1] webapp=/solr path=/select params={shards=127.0.0.1:14355/solr|[ff01::213]:33332/solr&sort=id+desc&shards.qt=tvrh&q=id:[3+TO+6]&tv=true&tv.all=true&qt=tvrh&wt=javabin&version=2} hits=4 status=0 QTime=13 
[junit4:junit4]   2> 10634 T233 C0 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&f.test_basictv.tv.df=false&tv.offsets=true&tv.positions=true&distrib=false&tv.df=true&f.test_basictv.tv.tf_idf=false&wt=javabin&version=2&tv.tf=true&f.test_basictv.tv.tf=false&q=id:[3+TO+6]&tv=true&f.test_posofftv.tv.positions=false&tv.tf_idf=true&tv.fl=test_basictv,test_notv,test_postv,test_offtv,test_posofftv&qt=tvrh&f.test_offtv.tv.offsets=false} hits=4 status=0 QTime=2 
[junit4:junit4]   2> 10644 T231 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&tv.positions=true&distrib=false&f.test_basictv.tv.tf_idf=false&version=2&fl=id&shard.url=127.0.0.1:14355/solr&NOW=1351731540068&f.test_posofftv.tv.positions=false&qt=tvrh&fsv=true&f.test_basictv.tv.df=false&tv.offsets=true&tv.df=true&wt=javabin&rows=10&tv.tf=true&f.test_basictv.tv.tf=false&start=0&q=id:[3+TO+6]&tv=true&tv.tf_idf=true&tv.fl=test_basictv,test_notv,test_postv,test_offtv,test_posofftv&isShard=true&f.test_offtv.tv.offsets=false} hits=4 status=0 QTime=3 
[junit4:junit4]   2> 10651 T229 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&tv.positions=true&distrib=false&f.test_basictv.tv.tf_idf=false&version=2&shard.url=127.0.0.1:14355/solr&NOW=1351731540068&f.test_posofftv.tv.positions=false&qt=tvrh&f.test_basictv.tv.df=false&ids=3,6,5,4&tv.offsets=true&tv.df=true&wt=javabin&tv.tf=true&f.test_basictv.tv.tf=false&q=id:[3+TO+6]&tv=true&tv.tf_idf=true&tv.fl=test_basictv,test_notv,test_postv,test_offtv,test_posofftv&isShard=true&f.test_offtv.tv.offsets=false} status=0 QTime=3 
[junit4:junit4]   2> 10654 T230 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&f.test_basictv.tv.df=false&tv.offsets=true&tv.positions=true&tv.df=true&f.test_basictv.tv.tf_idf=false&wt=javabin&version=2&shards=127.0.0.1:14355/solr&tv.tf=true&f.test_basictv.tv.tf=false&q=id:[3+TO+6]&tv=true&f.test_posofftv.tv.positions=false&tv.tf_idf=true&tv.fl=test_basictv,test_notv,test_postv,test_offtv,test_posofftv&qt=tvrh&f.test_offtv.tv.offsets=false} hits=4 status=0 QTime=16 
[junit4:junit4]   2> 10666 T217 C0 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&tv.tf=true&shards.qt=tvrh&q=*:*&tv=true&distrib=false&qt=tvrh&wt=javabin&version=2} hits=10 status=0 QTime=7 
[junit4:junit4]   2> 10675 T229 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+desc&shards.qt=tvrh&distrib=false&wt=javabin&rows=10&version=2&fl=id&shard.url=[ff01::213]:33332/solr|[ff01::114]:33332/solr|[ff01::083]:33332/solr|127.0.0.1:14355/solr&NOW=1351731540102&tv.tf=true&start=0&q=*:*&tv=true&isShard=true&qt=tvrh&fsv=true} hits=10 status=0 QTime=1 
[junit4:junit4]   2> 10684 T230 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&ids=3,2,1,0,7,6,5,4,9,8&distrib=false&wt=javabin&version=2&shard.url=[ff01::213]:33332/solr|[ff01::114]:33332/solr|[ff01::083]:33332/solr|127.0.0.1:14355/solr&NOW=1351731540102&tv.tf=true&q=*:*&tv=true&isShard=true&qt=tvrh} status=0 QTime=3 
[junit4:junit4]   2> 10688 T231 C1 REQ [collection1] webapp=/solr path=/select params={shards=[ff01::213]:33332/solr|[ff01::114]:33332/solr|[ff01::083]:33332/solr|127.0.0.1:14355/solr&sort=id+desc&tv.tf=true&shards.qt=tvrh&q=*:*&tv=true&qt=tvrh&wt=javabin&version=2} hits=10 status=0 QTime=16 
[junit4:junit4]   2> 10696 T216 C0 REQ [collection1] webapp=/solr path=/select params={fl=*,score&sort=id+asc&tv.tf=true&shards.qt=tvrh&q=*:*&tv=true&distrib=false&tv.fl=test_basictv,test_offtv&qt=tvrh&wt=javabin&version=2} hits=10 status=0 QTime=3 
[junit4:junit4]   2> 10706 T230 C1 REQ [collection1] webapp=/solr path=/select params={sort=id+asc&shards.qt=tvrh&distrib=false&wt=javabin&rows=10&version=2&fl=id,score&shard.url=[ff01::213]:33332/solr|127.0.0.1:14355/solr|[ff01::114]:33332/solr&NOW=1351731540131&tv.tf=true&start=0&q=*:*&tv=true&tv.fl=test_basictv,test_offtv&isShard=true&qt=tvrh&fsv=true} hits=10 status=0 QTime=2 
[junit4:junit4]   2> 10712 T231 C1 REQ [collection1] webapp=/solr path=/select params={shards.qt=tvrh&ids=3,2,1,0,7,6,5,4,9,8&distrib=false&wt=javabin&version=2&fl=*,score&shard.url=[ff01::213]:33332/solr|127.0.0.1:14355/solr|[ff01::114]:33332/solr&NOW=1351731540131&tv.tf=true&q=*:*&tv=true&tv.fl=test_basictv,test_offtv&isShard=true&qt=tvrh} status=0 QTime=2 
[junit4:junit4]   2> 10715 T229 C1 REQ [collection1] webapp=/solr path=/select params={fl=*,score&shards=[ff01::213]:33332/solr|127.0.0.1:14355/solr|[ff01::114]:33332/solr&sort=id+asc&tv.tf=true&shards.qt=tvrh&q=*:*&tv=true&tv.fl=test_basictv,test_offtv&qt=tvrh&wt=javabin&version=2} hits=10 status=0 QTime=14 
[junit4:junit4]   2> 10724 T214 C0 REQ [collection1] webapp=/

[...truncated too long message...]

nector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene41: {timestamp=PostingsFormat(name=MockRandom), range_facet_l=PostingsFormat(name=MockRandom), id=PostingsFormat(name=Direct), range_facet_sl=PostingsFormat(name=Direct), range_facet_si=PostingsFormat(name=MockRandom), teststop=PostingsFormat(name=MockRandom), multiDefault=PostingsFormat(name=MockRandom), intDefault=PostingsFormat(name=Asserting)}, sim=DefaultSimilarity, locale=mk_MK, timezone=America/Aruba
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_06 (64-bit)/cpus=16,threads=5,free=112184288,total=153092096
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestReload, UpdateRequestProcessorFactoryTest, SpellCheckComponentTest, SuggesterWFSTTest, JsonLoaderTest, LeaderElectionIntegrationTest, SolrCoreCheckLockOnStartupTest, AnalysisAfterCoreReloadTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=AnalysisAfterCoreReloadTest -Dtests.seed=6FAC95058473C9E4 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=mk_MK -Dtests.timezone=America/Aruba -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR   0.00s J0 | AnalysisAfterCoreReloadTest (suite) <<<
[junit4:junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.AnalysisAfterCoreReloadTest: 
[junit4:junit4]    >    1) Thread[id=156, name=qtp337577407-156 Acceptor1 SelectChannelConnector@0.0.0.0:14453, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=155, name=qtp337577407-155 Acceptor0 SelectChannelConnector@0.0.0.0:14453, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=158, name=qtp337577407-158 Acceptor3 SelectChannelConnector@0.0.0.0:14453, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=157, name=qtp337577407-157 Acceptor2 SelectChannelConnector@0.0.0.0:14453, state=RUNNABLE, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([6FAC95058473C9E4]:0)
[junit4:junit4]    > Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4]    >    1) Thread[id=156, name=qtp337577407-156 Acceptor1 SelectChannelConnector@0.0.0.0:14453, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=155, name=qtp337577407-155 Acceptor0 SelectChannelConnector@0.0.0.0:14453, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=158, name=qtp337577407-158 Acceptor3 SelectChannelConnector@0.0.0.0:14453, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=157, name=qtp337577407-157 Acceptor2 SelectChannelConnector@0.0.0.0:14453, state=RUNNABLE, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([6FAC95058473C9E4]:0)
[junit4:junit4] Completed on J0 in 28.49s, 1 test, 2 errors <<< FAILURES!

[...truncated 287 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:335: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/module-build.xml:61: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:1163: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:827: There were test failures: 233 suites, 965 tests, 4 suite-level errors, 845 ignored

Total time: 20 minutes 53 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure

[JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 616 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.

Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/616/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.TestDistributedGrouping

Error Message:
36 threads leaked from SUITE scope at org.apache.solr.TestDistributedGrouping:     1) Thread[id=167, name=qtp1514649449-167 Acceptor2 SelectChannelConnector@0.0.0.0:19908, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=88, name=qtp571201483-88 Acceptor3 SelectChannelConnector@0.0.0.0:19892, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=73, name=qtp1799456698-73 Acceptor2 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=39, name=qtp1340662491-39 Acceptor3 SelectChannelConnector@0.0.0.0:16514, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=155, name=qtp109963104-155 Acceptor3 SelectChannelConnector@0.0.0.0:19906, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=100, name=qtp1318450927-100 Acceptor2 SelectChannelConnector@0.0.0.0:19893, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=37, name=qtp1340662491-37 Acceptor1 SelectChannelConnector@0.0.0.0:16514, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=50, name=qtp1832089855-50 Acceptor0 SelectChannelConnector@0.0.0.0:19879, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=140, name=qtp29394673-140 Acceptor2 SelectChannelConnector@0.0.0.0:19905, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=124, name=qtp1017859891-124 Acceptor0 SelectChannelConnector@0.0.0.0:19904, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=87, name=qtp571201483-87 Acceptor2 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=154, name=qtp109963104-154 Acceptor2 SelectChannelConnector@0.0.0.0:19906, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=168, name=qtp1514649449-168 Acceptor3 SelectChannelConnector@0.0.0.0:19908, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=139, name=qtp29394673-139 Acceptor1 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=99, name=qtp1318450927-99 Acceptor1 SelectChannelConnector@0.0.0.0:19893, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=51, name=qtp1832089855-51 Acceptor1 SelectChannelConnector@0.0.0.0:19879, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=38, name=qtp1340662491-38 Acceptor2 SelectChannelConnector@0.0.0.0:16514, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=165, name=qtp1514649449-165 Acceptor0 SelectChannelConnector@0.0.0.0:19908, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=152, name=qtp109963104-152 Acceptor0 SelectChannelConnector@0.0.0.0:19906, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=126, name=qtp1017859891-126 Acceptor2 SelectChannelConnector@0.0.0.0:19904, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   21) Thread[id=153, name=qtp109963104-153 Acceptor1 SelectChannelConnector@0.0.0.0:19906, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   22) Thread[id=98, name=qtp1318450927-98 Acceptor0 SelectChannelConnector@0.0.0.0:19893, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   23) Thread[id=36, name=qtp1340662491-36 Acceptor0 SelectChannelConnector@0.0.0.0:16514, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   24) Thread[id=53, name=qtp1832089855-53 Acceptor3 SelectChannelConnector@0.0.0.0:19879, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   25) Thread[id=101, name=qtp1318450927-101 Acceptor3 SelectChannelConnector@0.0.0.0:19893, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   26) Thread[id=127, name=qtp1017859891-127 Acceptor3 SelectChannelConnector@0.0.0.0:19904, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   27) Thread[id=71, name=qtp1799456698-71 Acceptor0 SelectChannelConnector@0.0.0.0:19889, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   28) Thread[id=166, name=qtp1514649449-166 Acceptor1 SelectChannelConnector@0.0.0.0:19908, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   29) Thread[id=72, name=qtp1799456698-72 Acceptor1 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   30) Thread[id=141, name=qtp29394673-141 Acceptor3 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   31) Thread[id=86, name=qtp571201483-86 Acceptor1 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   32) Thread[id=85, name=qtp571201483-85 Acceptor0 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   33) Thread[id=52, name=qtp1832089855-52 Acceptor2 SelectChannelConnector@0.0.0.0:19879, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   34) Thread[id=138, name=qtp29394673-138 Acceptor0 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   35) Thread[id=125, name=qtp1017859891-125 Acceptor1 SelectChannelConnector@0.0.0.0:19904, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   36) Thread[id=74, name=qtp1799456698-74 Acceptor3 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 36 threads leaked from SUITE scope at org.apache.solr.TestDistributedGrouping: 
   1) Thread[id=167, name=qtp1514649449-167 Acceptor2 SelectChannelConnector@0.0.0.0:19908, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=88, name=qtp571201483-88 Acceptor3 SelectChannelConnector@0.0.0.0:19892, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=73, name=qtp1799456698-73 Acceptor2 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=39, name=qtp1340662491-39 Acceptor3 SelectChannelConnector@0.0.0.0:16514, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=155, name=qtp109963104-155 Acceptor3 SelectChannelConnector@0.0.0.0:19906, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=100, name=qtp1318450927-100 Acceptor2 SelectChannelConnector@0.0.0.0:19893, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=37, name=qtp1340662491-37 Acceptor1 SelectChannelConnector@0.0.0.0:16514, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=50, name=qtp1832089855-50 Acceptor0 SelectChannelConnector@0.0.0.0:19879, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=140, name=qtp29394673-140 Acceptor2 SelectChannelConnector@0.0.0.0:19905, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=124, name=qtp1017859891-124 Acceptor0 SelectChannelConnector@0.0.0.0:19904, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=87, name=qtp571201483-87 Acceptor2 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=154, name=qtp109963104-154 Acceptor2 SelectChannelConnector@0.0.0.0:19906, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=168, name=qtp1514649449-168 Acceptor3 SelectChannelConnector@0.0.0.0:19908, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=139, name=qtp29394673-139 Acceptor1 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=99, name=qtp1318450927-99 Acceptor1 SelectChannelConnector@0.0.0.0:19893, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=51, name=qtp1832089855-51 Acceptor1 SelectChannelConnector@0.0.0.0:19879, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=38, name=qtp1340662491-38 Acceptor2 SelectChannelConnector@0.0.0.0:16514, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=165, name=qtp1514649449-165 Acceptor0 SelectChannelConnector@0.0.0.0:19908, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=152, name=qtp109963104-152 Acceptor0 SelectChannelConnector@0.0.0.0:19906, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=126, name=qtp1017859891-126 Acceptor2 SelectChannelConnector@0.0.0.0:19904, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  21) Thread[id=153, name=qtp109963104-153 Acceptor1 SelectChannelConnector@0.0.0.0:19906, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  22) Thread[id=98, name=qtp1318450927-98 Acceptor0 SelectChannelConnector@0.0.0.0:19893, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  23) Thread[id=36, name=qtp1340662491-36 Acceptor0 SelectChannelConnector@0.0.0.0:16514, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  24) Thread[id=53, name=qtp1832089855-53 Acceptor3 SelectChannelConnector@0.0.0.0:19879, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  25) Thread[id=101, name=qtp1318450927-101 Acceptor3 SelectChannelConnector@0.0.0.0:19893, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  26) Thread[id=127, name=qtp1017859891-127 Acceptor3 SelectChannelConnector@0.0.0.0:19904, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  27) Thread[id=71, name=qtp1799456698-71 Acceptor0 SelectChannelConnector@0.0.0.0:19889, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  28) Thread[id=166, name=qtp1514649449-166 Acceptor1 SelectChannelConnector@0.0.0.0:19908, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  29) Thread[id=72, name=qtp1799456698-72 Acceptor1 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  30) Thread[id=141, name=qtp29394673-141 Acceptor3 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  31) Thread[id=86, name=qtp571201483-86 Acceptor1 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  32) Thread[id=85, name=qtp571201483-85 Acceptor0 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  33) Thread[id=52, name=qtp1832089855-52 Acceptor2 SelectChannelConnector@0.0.0.0:19879, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  34) Thread[id=138, name=qtp29394673-138 Acceptor0 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  35) Thread[id=125, name=qtp1017859891-125 Acceptor1 SelectChannelConnector@0.0.0.0:19904, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  36) Thread[id=74, name=qtp1799456698-74 Acceptor3 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([9A74CB360EB6D5D0]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.TestDistributedGrouping

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=167, name=qtp1514649449-167 Acceptor2 SelectChannelConnector@0.0.0.0:19908, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=88, name=qtp571201483-88 Acceptor3 SelectChannelConnector@0.0.0.0:19892, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=73, name=qtp1799456698-73 Acceptor2 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=39, name=qtp1340662491-39 Acceptor3 SelectChannelConnector@0.0.0.0:16514, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=155, name=qtp109963104-155 Acceptor3 SelectChannelConnector@0.0.0.0:19906, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=100, name=qtp1318450927-100 Acceptor2 SelectChannelConnector@0.0.0.0:19893, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=37, name=qtp1340662491-37 Acceptor1 SelectChannelConnector@0.0.0.0:16514, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=50, name=qtp1832089855-50 Acceptor0 SelectChannelConnector@0.0.0.0:19879, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=140, name=qtp29394673-140 Acceptor2 SelectChannelConnector@0.0.0.0:19905, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=124, name=qtp1017859891-124 Acceptor0 SelectChannelConnector@0.0.0.0:19904, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=87, name=qtp571201483-87 Acceptor2 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=154, name=qtp109963104-154 Acceptor2 SelectChannelConnector@0.0.0.0:19906, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=139, name=qtp29394673-139 Acceptor1 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=168, name=qtp1514649449-168 Acceptor3 SelectChannelConnector@0.0.0.0:19908, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=38, name=qtp1340662491-38 Acceptor2 SelectChannelConnector@0.0.0.0:16514, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=51, name=qtp1832089855-51 Acceptor1 SelectChannelConnector@0.0.0.0:19879, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=99, name=qtp1318450927-99 Acceptor1 SelectChannelConnector@0.0.0.0:19893, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=165, name=qtp1514649449-165 Acceptor0 SelectChannelConnector@0.0.0.0:19908, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=152, name=qtp109963104-152 Acceptor0 SelectChannelConnector@0.0.0.0:19906, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=126, name=qtp1017859891-126 Acceptor2 SelectChannelConnector@0.0.0.0:19904, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   21) Thread[id=98, name=qtp1318450927-98 Acceptor0 SelectChannelConnector@0.0.0.0:19893, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   22) Thread[id=153, name=qtp109963104-153 Acceptor1 SelectChannelConnector@0.0.0.0:19906, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   23) Thread[id=36, name=qtp1340662491-36 Acceptor0 SelectChannelConnector@0.0.0.0:16514, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   24) Thread[id=53, name=qtp1832089855-53 Acceptor3 SelectChannelConnector@0.0.0.0:19879, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   25) Thread[id=101, name=qtp1318450927-101 Acceptor3 SelectChannelConnector@0.0.0.0:19893, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   26) Thread[id=127, name=qtp1017859891-127 Acceptor3 SelectChannelConnector@0.0.0.0:19904, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   27) Thread[id=71, name=qtp1799456698-71 Acceptor0 SelectChannelConnector@0.0.0.0:19889, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   28) Thread[id=166, name=qtp1514649449-166 Acceptor1 SelectChannelConnector@0.0.0.0:19908, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   29) Thread[id=72, name=qtp1799456698-72 Acceptor1 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   30) Thread[id=141, name=qtp29394673-141 Acceptor3 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   31) Thread[id=86, name=qtp571201483-86 Acceptor1 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   32) Thread[id=85, name=qtp571201483-85 Acceptor0 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   33) Thread[id=52, name=qtp1832089855-52 Acceptor2 SelectChannelConnector@0.0.0.0:19879, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   34) Thread[id=138, name=qtp29394673-138 Acceptor0 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   35) Thread[id=125, name=qtp1017859891-125 Acceptor1 SelectChannelConnector@0.0.0.0:19904, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   36) Thread[id=74, name=qtp1799456698-74 Acceptor3 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=167, name=qtp1514649449-167 Acceptor2 SelectChannelConnector@0.0.0.0:19908, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=88, name=qtp571201483-88 Acceptor3 SelectChannelConnector@0.0.0.0:19892, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=73, name=qtp1799456698-73 Acceptor2 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=39, name=qtp1340662491-39 Acceptor3 SelectChannelConnector@0.0.0.0:16514, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=155, name=qtp109963104-155 Acceptor3 SelectChannelConnector@0.0.0.0:19906, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=100, name=qtp1318450927-100 Acceptor2 SelectChannelConnector@0.0.0.0:19893, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=37, name=qtp1340662491-37 Acceptor1 SelectChannelConnector@0.0.0.0:16514, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=50, name=qtp1832089855-50 Acceptor0 SelectChannelConnector@0.0.0.0:19879, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=140, name=qtp29394673-140 Acceptor2 SelectChannelConnector@0.0.0.0:19905, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=124, name=qtp1017859891-124 Acceptor0 SelectChannelConnector@0.0.0.0:19904, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=87, name=qtp571201483-87 Acceptor2 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=154, name=qtp109963104-154 Acceptor2 SelectChannelConnector@0.0.0.0:19906, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=139, name=qtp29394673-139 Acceptor1 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=168, name=qtp1514649449-168 Acceptor3 SelectChannelConnector@0.0.0.0:19908, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=38, name=qtp1340662491-38 Acceptor2 SelectChannelConnector@0.0.0.0:16514, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=51, name=qtp1832089855-51 Acceptor1 SelectChannelConnector@0.0.0.0:19879, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=99, name=qtp1318450927-99 Acceptor1 SelectChannelConnector@0.0.0.0:19893, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=165, name=qtp1514649449-165 Acceptor0 SelectChannelConnector@0.0.0.0:19908, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=152, name=qtp109963104-152 Acceptor0 SelectChannelConnector@0.0.0.0:19906, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=126, name=qtp1017859891-126 Acceptor2 SelectChannelConnector@0.0.0.0:19904, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  21) Thread[id=98, name=qtp1318450927-98 Acceptor0 SelectChannelConnector@0.0.0.0:19893, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  22) Thread[id=153, name=qtp109963104-153 Acceptor1 SelectChannelConnector@0.0.0.0:19906, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  23) Thread[id=36, name=qtp1340662491-36 Acceptor0 SelectChannelConnector@0.0.0.0:16514, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  24) Thread[id=53, name=qtp1832089855-53 Acceptor3 SelectChannelConnector@0.0.0.0:19879, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  25) Thread[id=101, name=qtp1318450927-101 Acceptor3 SelectChannelConnector@0.0.0.0:19893, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  26) Thread[id=127, name=qtp1017859891-127 Acceptor3 SelectChannelConnector@0.0.0.0:19904, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  27) Thread[id=71, name=qtp1799456698-71 Acceptor0 SelectChannelConnector@0.0.0.0:19889, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  28) Thread[id=166, name=qtp1514649449-166 Acceptor1 SelectChannelConnector@0.0.0.0:19908, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  29) Thread[id=72, name=qtp1799456698-72 Acceptor1 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  30) Thread[id=141, name=qtp29394673-141 Acceptor3 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  31) Thread[id=86, name=qtp571201483-86 Acceptor1 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  32) Thread[id=85, name=qtp571201483-85 Acceptor0 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  33) Thread[id=52, name=qtp1832089855-52 Acceptor2 SelectChannelConnector@0.0.0.0:19879, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  34) Thread[id=138, name=qtp29394673-138 Acceptor0 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  35) Thread[id=125, name=qtp1017859891-125 Acceptor1 SelectChannelConnector@0.0.0.0:19904, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  36) Thread[id=74, name=qtp1799456698-74 Acceptor3 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([9A74CB360EB6D5D0]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.update.SolrCmdDistributorTest

Error Message:
20 threads leaked from SUITE scope at org.apache.solr.update.SolrCmdDistributorTest:     1) Thread[id=65, name=qtp1096973217-65 Acceptor2 SelectChannelConnector@0.0.0.0:12325, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=49, name=qtp117369574-49 Acceptor0 SelectChannelConnector@0.0.0.0:33200, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=22, name=qtp1496026375-22 Acceptor1 SelectChannelConnector@0.0.0.0:58061, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=52, name=qtp117369574-52 Acceptor3 SelectChannelConnector@0.0.0.0:33200, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=77, name=qtp479278366-77 Acceptor0 SelectChannelConnector@0.0.0.0:35144, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=23, name=qtp1496026375-23 Acceptor2 SelectChannelConnector@0.0.0.0:58061, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=66, name=qtp1096973217-66 Acceptor3 SelectChannelConnector@0.0.0.0:12325, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=35, name=qtp316387877-35 Acceptor0 SelectChannelConnector@0.0.0.0:25933, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=21, name=qtp1496026375-21 Acceptor0 SelectChannelConnector@0.0.0.0:58061, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=64, name=qtp1096973217-64 Acceptor1 SelectChannelConnector@0.0.0.0:12325, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=36, name=qtp316387877-36 Acceptor1 SelectChannelConnector@0.0.0.0:25933, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=79, name=qtp479278366-79 Acceptor2 SelectChannelConnector@0.0.0.0:35144, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=63, name=qtp1096973217-63 Acceptor0 SelectChannelConnector@0.0.0.0:12325, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=51, name=qtp117369574-51 Acceptor2 SelectChannelConnector@0.0.0.0:33200, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=37, name=qtp316387877-37 Acceptor2 SelectChannelConnector@0.0.0.0:25933, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=38, name=qtp316387877-38 Acceptor3 SelectChannelConnector@0.0.0.0:25933, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=50, name=qtp117369574-50 Acceptor1 SelectChannelConnector@0.0.0.0:33200, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=78, name=qtp479278366-78 Acceptor1 SelectChannelConnector@0.0.0.0:35144, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=24, name=qtp1496026375-24 Acceptor3 SelectChannelConnector@0.0.0.0:58061, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=80, name=qtp479278366-80 Acceptor3 SelectChannelConnector@0.0.0.0:35144, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 20 threads leaked from SUITE scope at org.apache.solr.update.SolrCmdDistributorTest: 
   1) Thread[id=65, name=qtp1096973217-65 Acceptor2 SelectChannelConnector@0.0.0.0:12325, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=49, name=qtp117369574-49 Acceptor0 SelectChannelConnector@0.0.0.0:33200, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=22, name=qtp1496026375-22 Acceptor1 SelectChannelConnector@0.0.0.0:58061, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=52, name=qtp117369574-52 Acceptor3 SelectChannelConnector@0.0.0.0:33200, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=77, name=qtp479278366-77 Acceptor0 SelectChannelConnector@0.0.0.0:35144, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=23, name=qtp1496026375-23 Acceptor2 SelectChannelConnector@0.0.0.0:58061, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=66, name=qtp1096973217-66 Acceptor3 SelectChannelConnector@0.0.0.0:12325, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=35, name=qtp316387877-35 Acceptor0 SelectChannelConnector@0.0.0.0:25933, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=21, name=qtp1496026375-21 Acceptor0 SelectChannelConnector@0.0.0.0:58061, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=64, name=qtp1096973217-64 Acceptor1 SelectChannelConnector@0.0.0.0:12325, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=36, name=qtp316387877-36 Acceptor1 SelectChannelConnector@0.0.0.0:25933, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=79, name=qtp479278366-79 Acceptor2 SelectChannelConnector@0.0.0.0:35144, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=63, name=qtp1096973217-63 Acceptor0 SelectChannelConnector@0.0.0.0:12325, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=51, name=qtp117369574-51 Acceptor2 SelectChannelConnector@0.0.0.0:33200, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=37, name=qtp316387877-37 Acceptor2 SelectChannelConnector@0.0.0.0:25933, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=38, name=qtp316387877-38 Acceptor3 SelectChannelConnector@0.0.0.0:25933, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=50, name=qtp117369574-50 Acceptor1 SelectChannelConnector@0.0.0.0:33200, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=78, name=qtp479278366-78 Acceptor1 SelectChannelConnector@0.0.0.0:35144, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=24, name=qtp1496026375-24 Acceptor3 SelectChannelConnector@0.0.0.0:58061, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=80, name=qtp479278366-80 Acceptor3 SelectChannelConnector@0.0.0.0:35144, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([9A74CB360EB6D5D0]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.update.SolrCmdDistributorTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=65, name=qtp1096973217-65 Acceptor2 SelectChannelConnector@0.0.0.0:12325, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=49, name=qtp117369574-49 Acceptor0 SelectChannelConnector@0.0.0.0:33200, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=22, name=qtp1496026375-22 Acceptor1 SelectChannelConnector@0.0.0.0:58061, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=52, name=qtp117369574-52 Acceptor3 SelectChannelConnector@0.0.0.0:33200, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=23, name=qtp1496026375-23 Acceptor2 SelectChannelConnector@0.0.0.0:58061, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=77, name=qtp479278366-77 Acceptor0 SelectChannelConnector@0.0.0.0:35144, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=66, name=qtp1096973217-66 Acceptor3 SelectChannelConnector@0.0.0.0:12325, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=21, name=qtp1496026375-21 Acceptor0 SelectChannelConnector@0.0.0.0:58061, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=35, name=qtp316387877-35 Acceptor0 SelectChannelConnector@0.0.0.0:25933, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=64, name=qtp1096973217-64 Acceptor1 SelectChannelConnector@0.0.0.0:12325, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=36, name=qtp316387877-36 Acceptor1 SelectChannelConnector@0.0.0.0:25933, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=63, name=qtp1096973217-63 Acceptor0 SelectChannelConnector@0.0.0.0:12325, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=79, name=qtp479278366-79 Acceptor2 SelectChannelConnector@0.0.0.0:35144, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=51, name=qtp117369574-51 Acceptor2 SelectChannelConnector@0.0.0.0:33200, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=37, name=qtp316387877-37 Acceptor2 SelectChannelConnector@0.0.0.0:25933, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=38, name=qtp316387877-38 Acceptor3 SelectChannelConnector@0.0.0.0:25933, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=50, name=qtp117369574-50 Acceptor1 SelectChannelConnector@0.0.0.0:33200, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=78, name=qtp479278366-78 Acceptor1 SelectChannelConnector@0.0.0.0:35144, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=24, name=qtp1496026375-24 Acceptor3 SelectChannelConnector@0.0.0.0:58061, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=80, name=qtp479278366-80 Acceptor3 SelectChannelConnector@0.0.0.0:35144, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=65, name=qtp1096973217-65 Acceptor2 SelectChannelConnector@0.0.0.0:12325, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=49, name=qtp117369574-49 Acceptor0 SelectChannelConnector@0.0.0.0:33200, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=22, name=qtp1496026375-22 Acceptor1 SelectChannelConnector@0.0.0.0:58061, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=52, name=qtp117369574-52 Acceptor3 SelectChannelConnector@0.0.0.0:33200, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=23, name=qtp1496026375-23 Acceptor2 SelectChannelConnector@0.0.0.0:58061, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=77, name=qtp479278366-77 Acceptor0 SelectChannelConnector@0.0.0.0:35144, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=66, name=qtp1096973217-66 Acceptor3 SelectChannelConnector@0.0.0.0:12325, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=21, name=qtp1496026375-21 Acceptor0 SelectChannelConnector@0.0.0.0:58061, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=35, name=qtp316387877-35 Acceptor0 SelectChannelConnector@0.0.0.0:25933, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=64, name=qtp1096973217-64 Acceptor1 SelectChannelConnector@0.0.0.0:12325, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=36, name=qtp316387877-36 Acceptor1 SelectChannelConnector@0.0.0.0:25933, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=63, name=qtp1096973217-63 Acceptor0 SelectChannelConnector@0.0.0.0:12325, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=79, name=qtp479278366-79 Acceptor2 SelectChannelConnector@0.0.0.0:35144, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=51, name=qtp117369574-51 Acceptor2 SelectChannelConnector@0.0.0.0:33200, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=37, name=qtp316387877-37 Acceptor2 SelectChannelConnector@0.0.0.0:25933, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=38, name=qtp316387877-38 Acceptor3 SelectChannelConnector@0.0.0.0:25933, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=50, name=qtp117369574-50 Acceptor1 SelectChannelConnector@0.0.0.0:33200, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=78, name=qtp479278366-78 Acceptor1 SelectChannelConnector@0.0.0.0:35144, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=24, name=qtp1496026375-24 Acceptor3 SelectChannelConnector@0.0.0.0:58061, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=80, name=qtp479278366-80 Acceptor3 SelectChannelConnector@0.0.0.0:35144, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([9A74CB360EB6D5D0]:0)




Build Log:
[...truncated 8300 lines...]
[junit4:junit4] Suite: org.apache.solr.update.SolrCmdDistributorTest
[junit4:junit4]   2> 7 T14 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> 457 T14 oejs.Server.doStart jetty-8.1.7.v20120910
[junit4:junit4]   2> 754 T14 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:58061
[junit4:junit4]   2> 842 T14 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 843 T14 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 843 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 905 T14 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 906 T14 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 907 T14 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 908 T14 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 908 T14 oasc.CoreContainer.<init> New CoreContainer 1143585732
[junit4:junit4]   2> 910 T14 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 910 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 959 T14 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 1028 T14 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 1029 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 1031 T14 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 1032 T14 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 1102 T14 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 1162 T14 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 1163 T14 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 1214 T14 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1829 T14 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1843 T14 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1847 T14 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1867 T14 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1874 T14 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1879 T14 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1882 T14 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1885 T14 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1352461659291/control/data/
[junit4:junit4]   2> 1886 T14 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 1888 T14 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1352461659291/control/data/index/
[junit4:junit4]   2> 1889 T14 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1352461659291/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1890 T14 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1352461659291/control/data/index forceNew:false
[junit4:junit4]   2> 1900 T14 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1ea046df lockFactory=org.apache.lucene.store.NativeFSLockFactory@576288db),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1901 T14 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 1938 T14 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1963 T14 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1984 T14 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1990 T14 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2006 T14 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2013 T14 oass.SolrIndexSearcher.<init> Opening Searcher@89e364f main
[junit4:junit4]   2> 2014 T14 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 2081 T14 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2081 T14 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2093 T14 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 2094 T14 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 2094 T14 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 2095 T14 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 2095 T14 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 2096 T14 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 2096 T14 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 2097 T14 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 2098 T14 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 2148 T14 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 2704 T29 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@89e364f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2707 T14 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 2708 T14 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 2709 T14 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 2766 T14 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 2789 T14 oejs.Server.doStart jetty-8.1.7.v20120910
[junit4:junit4]   2> 2792 T14 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:25933
[junit4:junit4]   2> 2793 T14 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2794 T14 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 2794 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2830 T14 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 2831 T14 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2831 T14 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 2832 T14 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 2832 T14 oasc.CoreContainer.<init> New CoreContainer 746137641
[junit4:junit4]   2> 2833 T14 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2834 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2863 T14 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 2883 T14 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 2884 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 2886 T14 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 2887 T14 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 2942 T14 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 2993 T14 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 2994 T14 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 3041 T14 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 3602 T14 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 3616 T14 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 3619 T14 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 3640 T14 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 3645 T14 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 3650 T14 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 3652 T14 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 3655 T14 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1352461659291/shard0/data/
[junit4:junit4]   2> 3655 T14 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 3656 T14 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1352461659291/shard0/data/index/
[junit4:junit4]   2> 3657 T14 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1352461659291/shard0/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 3673 T14 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1352461659291/shard0/data/index forceNew:false
[junit4:junit4]   2> 3677 T14 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@1d5c9aa6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@320c4843)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 3677 T14 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 3679 T14 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 3680 T14 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 3680 T14 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 3681 T14 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 3691 T14 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 3697 T14 oass.SolrIndexSearcher.<init> Opening Searcher@6181ab1 main
[junit4:junit4]   2> 3697 T14 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3699 T14 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 3699 T14 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 3700 T14 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 3701 T14 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 3701 T14 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 3702 T14 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 3702 T14 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 3703 T14 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 3703 T14 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 3704 T14 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 3704 T14 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 3705 T14 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 3719 T43 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6181ab1 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 3719 T14 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 3720 T14 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 3721 T14 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3721 T14 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3732 T14 oejs.Server.doStart jetty-8.1.7.v20120910
[junit4:junit4]   2> 3735 T14 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:33200
[junit4:junit4]   2> 3736 T14 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3737 T14 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 3737 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 3766 T14 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 3767 T14 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3768 T14 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 3768 T14 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 3769 T14 oasc.CoreContainer.<init> New CoreContainer 167283382
[junit4:junit4]   2> 3769 T14 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 3770 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 3796 T14 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 3813 T14 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 3814 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 3816 T14 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 3817 T14 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 3855 T14 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 3898 T14 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 3900 T14 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 3939 T14 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 4492 T14 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 4511 T14 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 4514 T14 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 4534 T14 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4539 T14 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4544 T14 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4547 T14 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4550 T14 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1352461659291/shard1/data/
[junit4:junit4]   2> 4551 T14 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 4553 T14 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1352461659291/shard1/data/index/
[junit4:junit4]   2> 4554 T14 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1352461659291/shard1/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 4556 T14 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1352461659291/shard1/data/index forceNew:false
[junit4:junit4]   2> 4560 T14 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@79e70822 lockFactory=org.apache.lucene.store.NativeFSLockFactory@16cb436a),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 4561 T14 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 4563 T14 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 4564 T14 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 4565 T14 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 4566 T14 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 4582 T14 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 4593 T14 oass.SolrIndexSearcher.<init> Opening Searcher@43d06911 main
[junit4:junit4]   2> 4594 T14 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 4596 T14 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 4597 T14 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 4598 T14 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 4599 T14 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4599 T14 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 4600 T14 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4601 T14 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4602 T14 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4602 T14 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4603 T14 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4604 T14 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4605 T14 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 4628 T57 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@43d06911 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 4628 T14 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 4629 T14 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 4630 T14 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 4631 T14 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 4646 T14 oejs.Server.doStart jetty-8.1.7.v20120910
[junit4:junit4]   2> 4650 T14 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:12325
[junit4:junit4]   2> 4652 T14 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4653 T14 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 4653 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4698 T14 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 4699 T14 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4700 T14 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 4701 T14 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 4701 T14 oasc.CoreContainer.<init> New CoreContainer 1620817728
[junit4:junit4]   2> 4703 T14 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4704 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4739 T14 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4763 T14 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 4764 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 4765 T14 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4766 T14 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 4803 T14 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 4846 T14 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 4847 T14 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4880 T14 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 5410 T14 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 5423 T14 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 5427 T14 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5446 T14 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5451 T14 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5456 T14 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5458 T14 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5460 T14 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1352461659291/shard2/data/
[junit4:junit4]   2> 5461 T14 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 5462 T14 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1352461659291/shard2/data/index/
[junit4:junit4]   2> 5462 T14 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1352461659291/shard2/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5463 T14 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1352461659291/shard2/data/index forceNew:false
[junit4:junit4]   2> 5467 T14 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1c5ca6ca lockFactory=org.apache.lucene.store.NativeFSLockFactory@4199d4f9),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5467 T14 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 5469 T14 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 5469 T14 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 5470 T14 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 5470 T14 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 5480 T14 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 5487 T14 oass.SolrIndexSearcher.<init> Opening Searcher@1790423d main
[junit4:junit4]   2> 5487 T14 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 5489 T14 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 5489 T14 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 5490 T14 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 5490 T14 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 5491 T14 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 5491 T14 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 5492 T14 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 5492 T14 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 5493 T14 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 5493 T14 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 5494 T14 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 5494 T14 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 5507 T14 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 5507 T71 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1790423d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5508 T14 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 5508 T14 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 5509 T14 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 5520 T14 oejs.Server.doStart jetty-8.1.7.v20120910
[junit4:junit4]   2> 5522 T14 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:35144
[junit4:junit4]   2> 5524 T14 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 5524 T14 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 5525 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 5573 T14 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 5574 T14 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 5574 T14 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 5575 T14 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 5576 T14 oasc.CoreContainer.<init> New CoreContainer 795252340
[junit4:junit4]   2> 5576 T14 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 5577 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 5612 T14 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 5631 T14 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 5632 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 5634 T14 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 5635 T14 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 5679 T14 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 5733 T14 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 5734 T14 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 5760 T14 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 6329 T14 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 6349 T14 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 6353 T14 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 6372 T14 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6377 T14 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6381 T14 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6383 T14 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6385 T14 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Test

[...truncated too long message...]

hannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   27) Thread[id=71, name=qtp1799456698-71 Acceptor0 SelectChannelConnector@0.0.0.0:19889, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   28) Thread[id=166, name=qtp1514649449-166 Acceptor1 SelectChannelConnector@0.0.0.0:19908, state=BLOCKED, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   29) Thread[id=72, name=qtp1799456698-72 Acceptor1 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   30) Thread[id=141, name=qtp29394673-141 Acceptor3 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   31) Thread[id=86, name=qtp571201483-86 Acceptor1 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   32) Thread[id=85, name=qtp571201483-85 Acceptor0 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   33) Thread[id=52, name=qtp1832089855-52 Acceptor2 SelectChannelConnector@0.0.0.0:19879, state=BLOCKED, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   34) Thread[id=138, name=qtp29394673-138 Acceptor0 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   35) Thread[id=125, name=qtp1017859891-125 Acceptor1 SelectChannelConnector@0.0.0.0:19904, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   36) Thread[id=74, name=qtp1799456698-74 Acceptor3 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([9A74CB360EB6D5D0]:0)
[junit4:junit4] Completed on J1 in 51.15s, 1 test, 2 errors <<< FAILURES!

[...truncated 308 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:335: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/module-build.xml:61: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:1163: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:827: There were test failures: 233 suites, 970 tests, 4 suite-level errors, 872 ignored

Total time: 20 minutes 16 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure

[JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 615 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.

Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/615/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.AnalysisAfterCoreReloadTest

Error Message:
4 threads leaked from SUITE scope at org.apache.solr.AnalysisAfterCoreReloadTest:     1) Thread[id=92, name=qtp1156597185-92 Acceptor3 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=91, name=qtp1156597185-91 Acceptor2 SelectChannelConnector@0.0.0.0:29439, state=RUNNABLE, group=TGRP-AnalysisAfterCoreReloadTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=89, name=qtp1156597185-89 Acceptor0 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=90, name=qtp1156597185-90 Acceptor1 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.AnalysisAfterCoreReloadTest: 
   1) Thread[id=92, name=qtp1156597185-92 Acceptor3 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=91, name=qtp1156597185-91 Acceptor2 SelectChannelConnector@0.0.0.0:29439, state=RUNNABLE, group=TGRP-AnalysisAfterCoreReloadTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=89, name=qtp1156597185-89 Acceptor0 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=90, name=qtp1156597185-90 Acceptor1 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([EFEC412AFD1FA1E8]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.AnalysisAfterCoreReloadTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=92, name=qtp1156597185-92 Acceptor3 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=91, name=qtp1156597185-91 Acceptor2 SelectChannelConnector@0.0.0.0:29439, state=RUNNABLE, group=TGRP-AnalysisAfterCoreReloadTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=89, name=qtp1156597185-89 Acceptor0 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=90, name=qtp1156597185-90 Acceptor1 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=92, name=qtp1156597185-92 Acceptor3 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=91, name=qtp1156597185-91 Acceptor2 SelectChannelConnector@0.0.0.0:29439, state=RUNNABLE, group=TGRP-AnalysisAfterCoreReloadTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=89, name=qtp1156597185-89 Acceptor0 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=90, name=qtp1156597185-90 Acceptor1 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([EFEC412AFD1FA1E8]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.request.TestRemoteStreaming

Error Message:
4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming:     1) Thread[id=25, name=qtp1604840243-25 Acceptor0 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=27, name=qtp1604840243-27 Acceptor2 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=26, name=qtp1604840243-26 Acceptor1 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=28, name=qtp1604840243-28 Acceptor3 SelectChannelConnector@0.0.0.0:29337, state=RUNNABLE, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming: 
   1) Thread[id=25, name=qtp1604840243-25 Acceptor0 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=27, name=qtp1604840243-27 Acceptor2 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=26, name=qtp1604840243-26 Acceptor1 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=28, name=qtp1604840243-28 Acceptor3 SelectChannelConnector@0.0.0.0:29337, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([EFEC412AFD1FA1E8]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.request.TestRemoteStreaming

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=25, name=qtp1604840243-25 Acceptor0 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=27, name=qtp1604840243-27 Acceptor2 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=26, name=qtp1604840243-26 Acceptor1 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=28, name=qtp1604840243-28 Acceptor3 SelectChannelConnector@0.0.0.0:29337, state=RUNNABLE, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=25, name=qtp1604840243-25 Acceptor0 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=27, name=qtp1604840243-27 Acceptor2 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=26, name=qtp1604840243-26 Acceptor1 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=28, name=qtp1604840243-28 Acceptor3 SelectChannelConnector@0.0.0.0:29337, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([EFEC412AFD1FA1E8]:0)




Build Log:
[...truncated 8460 lines...]
[junit4:junit4] Suite: org.apache.solr.request.TestRemoteStreaming
[junit4:junit4]   2> 0 T18 oas.SolrTestCaseJ4.initCore ####initCore
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-TestRemoteStreaming-1352414376210
[junit4:junit4]   2> 1 T18 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4:junit4]   2> 375 T18 oejs.Server.doStart jetty-8.1.7.v20120910
[junit4:junit4]   2> 637 T18 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:29337
[junit4:junit4]   2> 675 T18 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 675 T18 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: solr/
[junit4:junit4]   2> 676 T18 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: 'solr/'
[junit4:junit4]   2> 711 T18 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 712 T18 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 712 T18 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: solr/
[junit4:junit4]   2> 713 T18 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solr/solr.xml
[junit4:junit4]   2> 713 T18 oasc.CoreContainer.<init> New CoreContainer 165061524
[junit4:junit4]   2> 714 T18 oasc.CoreContainer$Initializer.initialize no solr.xml file found - using default
[junit4:junit4]   2> 714 T18 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'solr/'
[junit4:junit4]   2> 715 T18 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'solr/'
[junit4:junit4]   2> 751 T18 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 795 T18 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: solr/collection1
[junit4:junit4]   2> 796 T18 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'solr/collection1/'
[junit4:junit4]   2> 870 T18 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 942 T18 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 946 T18 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 978 T18 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1694 T18 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1713 T18 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1718 T18 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1766 T18 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1787 T18 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1793 T18 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1801 T18 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1805 T18 oasc.SolrCore.<init> [collection1] Opening new SolrCore at solr/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-TestRemoteStreaming-1352414376210/
[junit4:junit4]   2> 1805 T18 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@70317b12
[junit4:junit4]   2> 1808 T18 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-TestRemoteStreaming-1352414376210/index/
[junit4:junit4]   2> 1808 T18 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-TestRemoteStreaming-1352414376210/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1815 T18 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-TestRemoteStreaming-1352414376210/index forceNew:false
[junit4:junit4]   2> 1818 T18 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1cb24b4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@d603051),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1819 T18 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 1820 T18 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 1830 T18 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1831 T18 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 1832 T18 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1833 T18 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 1834 T18 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1834 T18 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1835 T18 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1836 T18 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1837 T18 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1838 T18 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 1839 T18 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 1840 T18 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1841 T18 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 1842 T18 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 1843 T18 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 1844 T18 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1845 T18 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1846 T18 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1847 T18 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1848 T18 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1849 T18 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1850 T18 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1851 T18 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1852 T18 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 1853 T18 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 1866 T18 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1871 T18 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 1873 T18 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: []
[junit4:junit4]   2> 1877 T18 oass.SolrIndexSearcher.<init> Opening Searcher@166fa972 main
[junit4:junit4]   2> 1878 T18 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 1879 T18 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1879 T18 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1880 T18 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 1913 T18 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 1954 T18 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 1954 T18 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 1954 T18 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 1955 T18 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 1955 T18 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 1956 T18 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 1956 T18 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 1956 T18 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 1957 T18 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 1957 T18 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 1973 T33 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@166fa972 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1980 T18 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 1980 T18 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 1981 T18 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 1981 T18 oas.SolrJettyTestBase.createJetty Jetty Assigned Port#29337
[junit4:junit4]   2> 1987 T18 oas.SolrTestCaseJ4.setUp ###Starting testStreamUrl
[junit4:junit4]   2> 2033 T18 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C3 name=collection1 org.apache.solr.core.SolrCore@5c64f59
[junit4:junit4]   2> 2880 T31 C3 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1cb24b4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@d603051),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 2881 T31 C3 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 2883 T31 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1234]} 0 32
[junit4:junit4]   2> 3038 T32 C3 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 3041 T32 C3 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1cb24b4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@d603051),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1cb24b4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@d603051),segFN=segments_2,generation=2,filenames=[_0.tis, _0.nrm, _0.fnm, _0.tii, _0.frq, segments_2, _0.fdx, _0.si, _0.prx, _0.fdt]
[junit4:junit4]   2> 3042 T32 C3 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 3043 T32 C3 oass.SolrIndexSearcher.<init> Opening Searcher@54707929 main
[junit4:junit4]   2> 3044 T32 C3 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3045 T32 C3 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 3049 T33 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@54707929 main{StandardDirectoryReader(segments_2:3 _0(4.1):C1)}
[junit4:junit4]   2> 3050 T32 C3 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 12
[junit4:junit4]   2> 3143 T31 C3 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=60 
[junit4:junit4]   2> 3163 T31 C3 REQ [collection1] webapp=/solr path=/select params={wt=csv&q=*:*&fl=id} hits=1 status=0 QTime=6 
[junit4:junit4]   2> 3205 T32 C3 REQ [collection1] webapp=/solr path=/debug/dump params={wt=xml&stream.url=http://127.0.0.1:29337/solr/select?q%3D*:*%26fl%3Did%26wt%3Dcsv} status=0 QTime=46 
[junit4:junit4]   2> 3213 T18 oas.SolrTestCaseJ4.tearDown ###Ending testStreamUrl
[junit4:junit4]   2> 3251 T18 oas.SolrTestCaseJ4.setUp ###Starting testNoUrlAccess
[junit4:junit4]   2> 3257 T30 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1234]} 0 1
[junit4:junit4]   2> 3261 T32 C3 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 3267 T32 C3 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1cb24b4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@d603051),segFN=segments_2,generation=2,filenames=[_0.tis, _0.nrm, _0.fnm, _0.tii, _0.frq, segments_2, _0.fdx, _0.si, _0.prx, _0.fdt]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1cb24b4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@d603051),segFN=segments_3,generation=3,filenames=[_1.fdx, _1.prx, _1.tis, _1.frq, _1.fdt, _1.si, _1.tii, _1.fnm, _1.nrm, segments_3]
[junit4:junit4]   2> 3267 T32 C3 oasc.SolrDeletionPolicy.updateCommits newest commit = 3
[junit4:junit4]   2> 3270 T32 C3 oass.SolrIndexSearcher.<init> Opening Searcher@299a3d4 main
[junit4:junit4]   2> 3270 T32 C3 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3271 T32 C3 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 3275 T33 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@299a3d4 main{StandardDirectoryReader(segments_3:7 _1(4.1):C1)}
[junit4:junit4]   2> 3275 T32 C3 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 14
[junit4:junit4]   2> 3280 T31 C3 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 3285 T30 C3 REQ [collection1] webapp=/solr path=/select params={wt=javabin&stream.url=http://127.0.0.1:29337/solr/update?commit%3Dtrue%26stream.body%3D%253Cdelete%253E%253Cquery%253E*%253A*%253C%252Fquery%253E%253C%252Fdelete%253E&q=*:*&version=2} hits=1 status=0 QTime=0 
[junit4:junit4]   2> 3289 T32 C3 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 3291 T18 oas.SolrTestCaseJ4.tearDown ###Ending testNoUrlAccess
[junit4:junit4]   2> 3298 T18 oas.SolrTestCaseJ4.setUp ###Starting testMakeDeleteAllUrl
[junit4:junit4]   2> 3304 T31 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1234]} 0 2
[junit4:junit4]   2> 3309 T30 C3 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 3314 T30 C3 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1cb24b4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@d603051),segFN=segments_3,generation=3,filenames=[_1.fdx, _1.prx, _1.tis, _1.frq, _1.fdt, _1.si, _1.tii, _1.fnm, _1.nrm, segments_3]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1cb24b4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@d603051),segFN=segments_4,generation=4,filenames=[_2.si, _2.tis, _2.prx, _2.fdt, _2.frq, _2.tii, _2.fdx, _2.nrm, _2.fnm, segments_4]
[junit4:junit4]   2> 3314 T30 C3 oasc.SolrDeletionPolicy.updateCommits newest commit = 4
[junit4:junit4]   2> 3316 T30 C3 oass.SolrIndexSearcher.<init> Opening Searcher@6df8952f main
[junit4:junit4]   2> 3316 T30 C3 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3317 T30 C3 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 3321 T33 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6df8952f main{StandardDirectoryReader(segments_4:11 _2(4.1):C1)}
[junit4:junit4]   2> 3322 T30 C3 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 13
[junit4:junit4]   2> 3326 T32 C3 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=0 
[junit4:junit4]   2> 3330 T31 C3 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4:junit4]   2> 3331 T31 C3 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 3332 T31 C3 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1cb24b4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@d603051),segFN=segments_4,generation=4,filenames=[_2.si, _2.tis, _2.prx, _2.fdt, _2.frq, _2.tii, _2.fdx, _2.nrm, _2.fnm, segments_4]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1cb24b4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@d603051),segFN=segments_5,generation=5,filenames=[segments_5]
[junit4:junit4]   2> 3333 T31 C3 oasc.SolrDeletionPolicy.updateCommits newest commit = 5
[junit4:junit4]   2> 3334 T31 C3 oass.SolrIndexSearcher.<init> Opening Searcher@47ffa006 main
[junit4:junit4]   2> 3335 T31 C3 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3336 T31 C3 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 3338 T33 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@47ffa006 main{StandardDirectoryReader(segments_5:12)}
[junit4:junit4]   2> 3339 T31 C3 UPDATE [collection1] webapp=/solr path=/update params={commit=true&stream.body=<delete><query>*:*</query></delete>} {deleteByQuery=*:*,commit=} 0 10
[junit4:junit4]   2> 3344 T30 C3 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 3345 T18 oas.SolrTestCaseJ4.tearDown ###Ending testMakeDeleteAllUrl
[junit4:junit4]   2> 3353 T18 oas.SolrTestCaseJ4.setUp ###Starting testQtUpdateFails
[junit4:junit4]   2> 3358 T32 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1234]} 0 2
[junit4:junit4]   2> 3364 T31 C3 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 3366 T31 C3 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1cb24b4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@d603051),segFN=segments_5,generation=5,filenames=[segments_5]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1cb24b4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@d603051),segFN=segments_6,generation=6,filenames=[_3.fdx, _3.nrm, _3.tii, _3.frq, _3.si, _3.fdt, _3.prx, _3.fnm, segments_6, _3.tis]
[junit4:junit4]   2> 3367 T31 C3 oasc.SolrDeletionPolicy.updateCommits newest commit = 6
[junit4:junit4]   2> 3419 T31 C3 oass.SolrIndexSearcher.<init> Opening Searcher@67588300 main
[junit4:junit4]   2> 3419 T31 C3 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3420 T31 C3 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 3423 T33 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@67588300 main{StandardDirectoryReader(segments_6:14 _3(4.1):C1)}
[junit4:junit4]   2> 3424 T31 C3 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 60
[junit4:junit4]   2> 3428 T30 C3 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 3433 T18 oas.SolrTestCaseJ4.tearDown ###Ending testQtUpdateFails
[junit4:junit4]   2> 3470 T18 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=165061524
[junit4:junit4]   2> 3471 T18 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@5c64f59
[junit4:junit4]   2> 3486 T18 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=5,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=4,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4]   2> 3487 T18 oasc.SolrCore.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 3487 T18 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 3487 T18 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 3489 T18 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 3495 T18 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 3596 T18 oejut.QueuedThreadPool.doStop WARNING 4 threads could not be stopped
[junit4:junit4]   2> 3596 T18 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> 3607 T17 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 4 leaked thread(s).
[junit4:junit4]   2> 23715 T17 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming: 
[junit4:junit4]   2> 	   1) Thread[id=25, name=qtp1604840243-25 Acceptor0 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   2) Thread[id=27, name=qtp1604840243-27 Acceptor2 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   3) Thread[id=26, name=qtp1604840243-26 Acceptor1 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   4) Thread[id=28, name=qtp1604840243-28 Acceptor3 SelectChannelConnector@0.0.0.0:29337, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 23716 T17 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
[junit4:junit4]   2> 	   1) Thread[id=25, name=qtp1604840243-25 Acceptor0 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	   2) Thread[id=27, name=qtp1604840243-27 Acceptor2 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	   3) Thread[id=26, name=qtp1604840243-26 Acceptor1 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	   4) Thread[id=28, name=qtp1604840243-28 Acceptor3 SelectChannelConnector@0.0.0.0:29337, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 26761 T17 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
[junit4:junit4]   2> 	   1) Thread[id=25, name=qtp1604840243-25 Acceptor0 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   2) Thread[id=27, name=qtp1604840243-27 Acceptor2 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   3) Thread[id=26, name=qtp1604840243-26 Acceptor1 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   4) Thread[id=28, name=qtp1604840243-28 Acceptor3 SelectChannelConnector@0.0.0.0:29337, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene3x, sim=DefaultSimilarity, locale=es_DO, timezone=Europe/Belgrade
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=5,free=62022968,total=130023424
[junit4:junit4]   2> NOTE: All tests run in this JVM: [SuggesterTSTTest, BasicFunctionalityTest, TestRemoteStreaming]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestRemoteStreaming -Dtests.seed=EFEC412AFD1FA1E8 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_DO -Dtests.timezone=Europe/Belgrade -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR   0.00s J1 | TestRemoteStreaming (suite) <<<
[junit4:junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming: 
[junit4:junit4]    >    1) Thread[id=25, name=qtp1604840243-25 Acceptor0 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=27, name=qtp1604840243-27 Acceptor2 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=26, name=qtp1604840243-26 Acceptor1 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=28, name=qtp1604840243-28 Acceptor3 SelectChannelConnector@0.0.0.0:29337, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([EFEC412AFD1FA1E8]:0)
[junit4:junit4]    > Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4]    >    1) Thread[id=25, name=qtp1604840243-25 Acceptor0 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=27, name=qtp1604840243-27 Acceptor2 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=26, name=qtp1604840243-26 Acceptor1 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=28, name=qtp1604840243-28 Acceptor3 SelectChannelConnector@0.0.0.0:29337, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([EFEC412AFD1FA1E8]:0)
[junit4:junit4] Completed on J1 in 26.91s, 4 tests, 2 errors <<< FAILURES!

[...truncated 523 lines...]
[junit4:junit4] Suite: org.apache.solr.AnalysisAfterCoreReloadTest
[junit4:junit4]   2> 5163 T81 oasu.AbstractSolrTestCase.setUp ####SETUP_START testStopwordsAfterCoreReload
[junit4:junit4]   2> 5165 T81 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1/'
[junit4:junit4]   2> 5167 T81 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solr-test-home-21381301548538531/collection1/lib/README' to classloader
[junit4:junit4]   2> 5168 T81 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solr-test-home-21381301548538531/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 5260 T81 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 5331 T81 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 5332 T81 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 5341 T81 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 5983 T81 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 5995 T81 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 5999 T81 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 6014 T81 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6019 T81 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6024 T81 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6025 T81 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6026 T81 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 6026 T81 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531
[junit4:junit4]   2> 6027 T81 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/'
[junit4:junit4]   2> 6039 T81 oasc.CoreContainer.<init> New CoreContainer 75451581
[junit4:junit4]   2> 6039 T81 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.AnalysisAfterCoreReloadTest-1352414417679/
[junit4:junit4]   2> 6040 T81 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@682efa99
[junit4:junit4]   2> 6041 T81 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.AnalysisAfterCoreReloadTest-1352414417679/index/
[junit4:junit4]   2> 6041 T81 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.AnalysisAfterCoreReloadTest-1352414417679/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 6043 T81 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.AnalysisAfterCoreReloadTest-1352414417679/index forceNew:false
[junit4:junit4]   2> 6046 T81 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5fc52571 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3c22c7c3),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 6047 T81 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 6048 T81 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 6051 T81 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 6052 T81 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 6052 T81 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 6053 T81 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 6054 T81 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 6055 T81 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 6055 T81 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 6056 T81 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 6057 T81 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 6058 T81 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 6059 T81 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 6060 T81 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 6060 T81 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 6061 T81 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 6061 T81 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 6062 T81 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 6062 T81 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6063 T81 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6064 T81 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6064 T81 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6065 T81 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6065 T81 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6066 T81 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6067 T81 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 6068 T81 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 6078 T81 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 6081 T81 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 6083 T81 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [THROW.ERROR.ON.ADD.UPDATEPROCESSOR.JS, SOLRCONFIG-HIGHLIGHT.XML, SCHEMA-REQUIRED-FIELDS.XML, SCHEMA-REPLICATION2.XML, TRIVIAL.UPDATEPROCESSOR1.JS, SCHEMA-MINIMAL.XML, BAD-SCHEMA-DUP-DYNAMICFIELD.XML, SOLRCONFIG-CACHING.XML, SOLRCONFIG-REPEATER.XML, BAD-SCHEMA-NONTEXT-ANALYZER.XML, CURRENCY.XML, SOLRCONFIG-MERGEPOLICY.XML, SOLRCONFIG-TLOG.XML, SOLRCONFIG-MASTER.XML, BAD-SCHEMA-UNIQUEKEY-MULTIVALUED.XML, SCHEMA11.XML, SOLRCONFIG-BASIC.XML, DA_COMPOUNDDICTIONARY.TXT, MISSLEADING.EXTENSION.UPDATEPROCESSOR.JS.TXT, SCHEMA-COPYFIELD-TEST.XML, SOLRCONFIG-SLAVE.XML, SOLRCONFIG-PROPINJECT-INDEXDEFAULT.XML, ELEVATE.XML, TRIVIAL.UPDATEPROCESSOR0.JS, BAD-SOLRCONFIG-BOGUS-SCRIPTENGINE-NAME.XML, SCHEMA-IB.XML, SCHEMA-CHARFILTERS.XML, SOLRCONFIG-QUERYSENDER.XML, DA_UTF8.XML, SCHEMA-REPLICATION1.XML, SOLRCONFIG-SNIPPET-PROCESSOR.XML, CONDITIONAL.UPDATEPROCESSOR.JS, MISSING.FUNCTIONS.UPDATEPROCESSOR.JS, SOLRCONFIG-ENABLEPLUGIN.XML, HYPHENATION.DTD, STEMDICT.TXT, SCHEMA-PHRASESUGGEST.XML, HUNSPELL-TEST.AFF, SCHEMA-SNIPPET-TYPE.XML, STOPTYPES-1.TXT, STOPWORDSWRONGENCODING.TXT, SCHEMA-NUMERIC.XML, SOLRCONFIG-TRANSFORMERS.XML, SOLRCONFIG-PROPINJECT.XML, BAD-SCHEMA-NOT-INDEXED-BUT-TF.XML, SOLRCONFIG-SIMPLELOCK.XML, WDFTYPES.TXT, SCHEMA-REVERSED.XML, STOPTYPES-2.TXT, BAD-SCHEMA-CURRENCY-FT-MULTIVALUED.XML, SOLRCONFIG-SPELLCHECKCOMPONENT.XML, BAD-SCHEMA-CURRENCY-MULTIVALUED.XML, SCHEMA-DFR.XML, SOLRCONFIG-PHRASESUGGEST.XML, BAD-SCHEMA-NOT-INDEXED-BUT-POS.XML, KEEP-1.TXT, OPEN-EXCHANGE-RATES.JSON, SCHEMA-SPATIAL.XML, STOPWITHBOM.TXT, SOLRCONFIG-SPELLCHECKER.XML, SCHEMA-BINARYFIELD.XML, SOLRCONFIG-UPDATE-PROCESSOR-CHAINS.XML, BAD-SCHEMA-OMIT-TF-BUT-NOT-POS.XML, SCHEMA-XINCLUDE.XML, BAD-SCHEMA-DUP-FIELDTYPE.XML, SOLRCONFIG-MASTER1.XML, SYNONYMS.TXT, BAD-SCHEMA-CURRENCY-DYNAMIC-MULTIVALUED.XML, SCHEMA_CODEC.XML, SCHEMA.XML, SOLRCONFIG-SOLR-749.XML, SOLRCONFIG-MASTER1-KEEPONEBACKUP.XML, STOP-2.TXT, SOLRCONFIG-FUNCTIONQUERY.XML, SCHEMA-LMDIRICHLET.XML, SOLRCONFIG-TERMINDEX.XML, SOLRCONFIG-ELEVATE.XML, STOPWORDS.TXT, SCHEMA-FOLDING.XML, SCHEMA-STOP-KEEP.XML, BAD-SCHEMA-NOT-INDEXED-BUT-NORMS.XML, SOLRCONFIG-SOLCOREPROPERTIES.XML, STOP-1.TXT, SOLRCONFIG-MASTER2.XML, SCHEMA-SPELLCHECKER.XML, SOLRCONFIG-RESPONSE-LOG-COMPONENT.XML, SOLRCONFIG-LAZYWRITER.XML, SCHEMA-LUCENEMATCHVERSION.XML, BAD-MP-SOLRCONFIG.XML, FRENCHARTICLES.TXT, SCHEMA15.XML, SOLRCONFIG-REQHANDLER.INCL, SCHEMASURROUND.XML, SCHEMA-COLLATEFILTER.XML, SOLRCONFIG-MASTER3.XML, HUNSPELL-TEST.DIC, BAD-SCHEMA-CODEC-GLOBAL-VS-FT-MISMATCH.XML, SOLRCONFIG-XINCLUDE.XML, SOLRCONFIG-DELPOLICY1.XML, SOLRCONFIG-SLAVE1.XML, SCHEMA-SIM.XML, SCHEMA-SNIPPET-FIELD.XML, SCHEMA-COLLATE.XML, STOP-SNOWBALL.TXT, BAD-SCHEMA-SIM-GLOBAL-VS-FT-MISMATCH.XML, PROTWORDS.TXT, SCHEMA-TRIE.XML, SOLRCONFIG_CODEC.XML, BAD-SOLRCONFIG-INVALID-SCRIPTFILE.XML, JASUGGEST.TXT, SCHEMA-TFIDF.XML, SOLRCONFIG-SCRIPT-UPDATEPROCESSOR.XML, SCHEMA-LMJELINEKMERCER.XML, PHRASESUGGEST.TXT, BAD-SOLRCONFIG-MISSING-SCRIPTFILE.XML, SOLRCONFIG-BASIC-LUCENEVERSION31.XML, BAD-SCHEMA-UNIQUEKEY-USES-DEFAULT.XML, OLD_SYNONYMS.TXT, SOLRCONFIG-DELPOLICY2.XML, XSLT, SOLRCONFIG-NATIVELOCK.XML, BAD-SCHEMA-DUP-FIELD.XML, SOLRCONFIG-NOCACHE.XML, SCHEMA-BM25.XML, ADDFIELDS.UPDATEPROCESSOR.JS, SOLRCONFIG-QUERYSENDER-NOQUERY.XML, SOLRCONFIG-ALTDIRECTORY.XML, COMPOUNDDICTIONARY.TXT, SOLRCONFIG-INDEXCONFIG.XML, SOLRCONFIG_PERF.XML, BAD-SCHEMA-ANALYZER-CLASS-AND-NESTED.XML, SCHEMA-NOT-REQUIRED-UNIQUE-KEY.XML, KEEP-2.TXT, BAD-SCHEMA-UNIQUEKEY-IS-COPYFIELD-DEST.XML, SCHEMA12.XML, MAPPING-ISOLATIN1ACCENT.TXT, BAD_SOLRCONFIG.XML, BAD-SCHEMA-EXTERNAL-FILEFIELD.XML]
[junit4:junit4]   2> 6085 T81 oass.SolrIndexSearcher.<init> Opening Searcher@1dd0a8c0 main
[junit4:junit4]   2> 6085 T81 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 6086 T81 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 6086 T81 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 6086 T81 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 6097 T81 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 6147 T81 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 6148 T81 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 6148 T81 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 6148 T81 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 6149 T81 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 6149 T81 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 6149 T81 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 6149 T81 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 6150 T81 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 6150 T81 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 6163 T82 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1dd0a8c0 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 6166 T81 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 6175 T81 oasu.AbstractSolrTestCase.setUp ####SETUP_END testStopwordsAfterCoreReload
[junit4:junit4]   2> 6569 T81 oejs.Server.doStart jetty-8.1.7.v20120910
[junit4:junit4]   2> 6827 T81 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:29439
[junit4:junit4]   2> 6864 T81 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 6865 T81 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531
[junit4:junit4]   2> 6865 T81 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/'
[junit4:junit4]   2> 6898 T81 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 6898 T81 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 6899 T81 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531
[junit4:junit4]   2> 6899 T81 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/solr.xml
[junit4:junit4]   2> 6899 T81 oasc.CoreContainer.<init> New CoreContainer 1932761421
[junit4:junit4]   2> 6900 T81 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/'
[junit4:junit4]   2> 6900 T81 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/'
[junit4:junit4]   2> 6924 T81 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 6940 T81 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1
[junit4:junit4]   2> 6940 T81 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1/'
[junit4:junit4]   2> 6941 T81 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solr-test-home-21381301548538531/collection1/lib/README' to classloader
[junit4:junit4]   2> 6942 T81 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solr-test-home-21381301548538531/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 6979 T81 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 7028 T81 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 7029 T81 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 7035 T81 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 7508 T81 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 7517 T81 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 7520 T81 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 7530 T81 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 7534 T81 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 7538 T81 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 7538 T81 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 7539 T81 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1/data/
[junit4:junit4]   2> 7540 T81 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@682efa99
[junit4:junit4]   2> 7541 T81 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1/data/index/
[junit4:junit4]   2> 7541 T81 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 7542 T81 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1/data/index forceNew:false
[junit4:junit4]   2> 7543 T81 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@680e6039 lockFactory=org.apache.lucene.store.NativeFSLockFactory@60e5272b; maxCacheMB=0.8359375 maxMergeSizeMB=0.57421875))),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 7544 T81 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 7544 T81 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 7547 T81 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 7547 T81 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 7547 T81 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 7548 T81 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 7548 T81 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 7549 T81 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 7549 T81 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 7550 T81 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 7550 T81 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 7551 T81 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 7552 T81 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 7552 T81 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 7553 T81 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 7553 T81 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 7553 T81 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 7554 T81 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 7554 T81 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7555 T81 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7555 T81 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7556 T81 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7556 T81 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7557 T81 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7558 T81 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7558 T81 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 7559 T81 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 7567 T81 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 7570 T81 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 7572 T81 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [THROW.ERROR.ON.ADD.UPDATEPROCESSOR.JS, SOLRCONFIG-HIGHLIGHT.XML, SCHEMA-REQUIRED-FIELDS.XML, SCHEMA-REPLICATION2.XML, TRIVIAL.UPDATEPROCESSOR1.JS, SCHEMA-MINIMAL.XML, BAD-SCHEMA-DUP-DYNAMICFIELD.XML, SOLRCONFIG-CACHING.XML, SOLRCONFIG-REPEATER.XML, BAD-SCHEMA-NONTEXT-ANALYZER.XML, CURRENCY.XML, SOLRCONFIG-MERGEPOLICY.XML, SOLRCONFIG-TLOG.XML, SOLRCONFIG-MASTER.XML, BAD-SCHEMA-UNIQUEKEY-MULTIVALUED.XML, SCHEMA11.XML, SOLRCONFIG-BASIC.XML, DA_COMPOUNDDICTIONARY.TXT, MISSLEADING.EXTENSION.UPDATEPROCESSOR.JS.TXT, SCHEMA-COPYFIELD-TEST.XML, SOLRCONFIG-SLAVE.XML, SOLRCONFIG-PROPINJECT-INDEXDEFAULT.XML, ELEVATE.XML, TRIVIAL.UPDATEPROCESSOR0.JS, BAD-SOLRCONFIG-BOGUS-SCRIPTENGINE-NAME.XML, SCHEMA-IB.XML, SCHEMA-CHARFILTERS.XML, SOLRCONFIG-QUERYSENDER.XML, DA_UTF8.XML, SCHEMA-REPLICATION1.XML, SOLRCONFIG-SNIPPET-PROCESSOR.XML, CONDITIONAL.UPDATEPROCESSOR.JS, MISSING.FUNCTIONS.UPDATEPROCESSOR.JS, SOLRCONFIG-ENABLEPLUGIN.XML, HYPHENATION.DTD, STEMDICT.TXT, SCHEMA-PHRASESUGGEST.XML, HUNSPELL-TEST.AFF, SCHEMA-SNIPPET-TYPE.XML, STOPTYPES-1.TXT, STOPWORDSWRONGENCODING.TXT, SCHEMA-NUMERIC.XML, SOLRCONFIG-TRANSFORMERS.XML, SOLRCONFIG-PROPINJECT.XML, BAD-SCHEMA-NOT-INDEXED-BUT-TF.XML, SOLRCONFIG-SIMPLELOCK.XML, WDFTYPES.TXT, SCHEMA-REVERSED.XML, STOPTYPES-2.TXT, BAD-SCHEMA-CURRENCY-FT-MULTIVALUED.XML, SOLRCONFIG-SPELLCHECKCOMPONENT.XML, BAD-SCHEMA-CURRENCY-MULTIVALUED.XML, SCHEMA-DFR.XML, SOLRCONFIG-PHRASESUGGEST.XML, BAD-SCHEMA-NOT-INDEXED-BUT-POS.XML, KEEP-1.TXT, OPEN-EXCHANGE-RATES.JSON, SCHEMA-SPATIAL.XML, STOPWITHBOM.TXT, SOLRCONFIG-SPELLCHECKER.XML, SCHEMA-BINARYFIELD.XML, SOLRCONFIG-UPDATE-PROCESSOR-CHAINS.XML, BAD-SCHEMA-OMIT-TF-BUT-NOT-POS.XML, SCHEMA-XINCLUDE.XML, BAD-SCHEMA-DUP-FIELDTYPE.XML, SOLRCONFIG-MASTER1.XML, SYNONYMS.TXT, BAD-SCHEMA-CURRENCY-DYNAMIC-MULTIVALUED.XML, SCHEMA_CODEC.XML, SCHEMA.XML, SOLRCONFIG-SOLR-749.XML, SOLRCONFIG-MASTER1-KEEPONEBACKUP.XML, STOP-2.TXT, SOLRCONFIG-FUNCTIONQUERY.XML, SCHEMA-LMDIRICHLET.XML, SOLRCONFIG-TERMINDEX.XML, SOLRCONFIG-ELEVATE.XML, STOPWORDS.TXT, SCHEMA-FOLDING.XML, SCHEMA-STOP-KEEP.XML, BAD-SCHEMA-NOT-INDEXED-BUT-NORMS.XML, SOLRCONFIG-SOLCOREPROPERTIES.XML, STOP-1.TXT, SOLRCONFIG-MASTER2.XML, SCHEMA-SPELLCHECKER.XML, SOLRCONFIG-RESPONSE-LOG-COMPONENT.XML, SOLRCONFIG-LAZYWRITER.XML, SCHEMA-LUCENEMATCHVERSION.XML, BAD-MP-SOLRCONFIG.XML, FRENCHARTICLES.TXT, SCHEMA15.XML, SOLRCONFIG-REQHANDLER.INCL, SCHEMASURROUND.XML, SCHEMA-COLLATEFILTER.XML, SOLRCONFIG-MASTER3.XML, HUNSPELL-TEST.DIC, BAD-SCHEMA-CODEC-GLOBAL-VS-FT-MISMATCH.XML, SOLRCONFIG-XINCLUDE.XML, SOLRCONFIG-DELPOLICY1.XML, SOLRCONFIG-SLAVE1.XML, SCHEMA-SIM.XML, SCHEMA-SNIPPET-FIELD.XML, SCHEMA-COLLATE.XML, STOP-SNOWBALL.TXT, BAD-SCHEMA-SIM-GLOBAL-VS-FT-MISMATCH.XML, PROTWORDS.TXT, SCHEMA-TRIE.XML, SOLRCONFIG_CODEC.XML, BAD-SOLRCONFIG-INVALID-SCRIPTFILE.XML, JASUGGEST.TXT, SCHEMA-TFIDF.XML, SOLRCONFIG-SCRIPT-UPDATEPROCESSOR.XML, SCHEMA-LMJELINEKMERCER.XML, PHRASESUGGEST.TXT, BAD-SOLRCONFIG-MISSING-SCRIPTFILE.XML, SOLRCONFIG-BASIC-LUCENEVERSION31.XML, BAD-SCHEMA-UNIQUEKEY-USES-DEFAULT.XML, OLD_SYNONYMS.TXT, SOLRCONFIG-DELPOLICY2.XML, XSLT, SOLRCONFIG-NATIVELOCK.XML, BAD-SCHEMA-DUP-FIELD.XML, SOLRCONFIG-NOCACHE.XML, SCHEMA-BM25.XML, ADDFIELDS.UPDATEPROCESSOR.JS, SOLRCONFIG-QUERYSENDER-NOQUERY.XML, SOLRCONFIG-ALTDIRECTORY.XML, COMPOUNDDICTIONARY.TXT, SOLRCONFIG-INDEXCONFIG.XML, SOLRCONFIG_PERF.XML, BAD-SCHEMA-ANALYZER-CLASS-AND-NESTED.XML, SCHEMA-NOT-REQUIRED-UNIQUE-KEY.XML, KEEP-2.TXT, BAD-SCHEMA-UNIQUEKEY-IS-COPYFIELD-DEST.XML, SCHEMA12.XML, MAPPING-ISOLATIN1ACCENT.TXT, BAD_SOLRCONFIG.XML, BAD-SCHEMA-EXTERNAL-FILEFIELD.XML]
[junit4:junit4]   2> 7574 T81 oass.SolrIndexSearcher.<init> Opening Searcher@19ccda74 main
[junit4:junit4]   2> 7575 T81 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 7575 T81 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 7575 T81 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 7576 T81 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 7578 T81 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 7588 T81 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 7588 T81 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 7589 T81 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 7589 T81 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 7589 T81 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 7590 T81 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 7590 T81 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 7590 T81 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 7591 T81 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 7591 T81 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 7605 T97 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@19ccda74 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 7610 T81 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 7611 T81 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 7611 T81 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 7696 T81 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C14 name=collection1 org.apache.solr.core.SolrCore@3d88eb46
[junit4:junit4]   2> 8668 T96 C14 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@680e6039 lockFactory=org.apache.lucene.store.NativeFSLockFactory@60e5272b; maxCacheMB=0.8359375 maxMergeSizeMB=0.57421875))),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 8669 T96 C14 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 8671 T96 C14 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 8719 T96 C14 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@680e6039 lockFactory=org.apache.lucene.store.NativeFSLockFactory@60e5272b; maxCacheMB=0.8359375 maxMergeSizeMB=0.57421875))),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@680e6039 lockFactory=org.apache.lucene.store.NativeFSLockFactory@60e5272b; maxCacheMB=0.8359375 maxMergeSizeMB=0.57421875))),segFN=segments_2,generation=2,filenames=[_0.fnm, _0_TestBloomFilteredLucene41Postings_0.blm, _0_MockSep_0.pos, _0_Lucene41_0.pos, _0_TestBloomFilteredLucene41Postings_0.tim, _0_nrm.cfs, _0_TestBloomFilteredLucene41Postings_0.tip, _0_TestBloomFilteredLucene41Postings_0.pos, _0_MockSep_0.tib, _0_Lucene41_0.doc, _0_MockSep_0.skp, _0_nrm.cfe, _0_TestBloomFilteredLucene41Postings_0.doc, _0_MockSep_0.frq, segments_2, _0.fdx, _0_MockSep_0.tii, _0.si, _0_MockSep_0.pyl, _0_MockSep_0.doc, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip]
[junit4:junit4]   2> 8720 T96 C14 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 8778 T96 C14 oass.SolrIndexSearcher.<init> Opening Searcher@6cb320ac main
[junit4:junit4]   2> 8779 T96 C14 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 8780 T96 C14 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 8781 T97 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6cb320ac main{StandardDirectoryReader(segments_2:3 _0(4.1):C1)}
[junit4:junit4]   2> 8782 T96 C14 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {add=[42],commit=} 0 140
[junit4:junit4]   2> 8921 T81 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9025 T94 C14 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=teststop:terma&version=2} hits=1 status=0 QTime=57 
[junit4:junit4]   2> 9027 T81 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9039 T95 C14 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=teststop:stopworda&version=2} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 9040 T81 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9051 T96 C14 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=teststop:stopwordb&version=2} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 9051 T81 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9060 T95 C14 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=teststop:stopwordc&version=2} hits=1 status=0 QTime=0 
[junit4:junit4]   2> 9062 T81 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9081 T96 oasc.CoreContainer.reload Reloading SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1
[junit4:junit4]   2> 9082 T96 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1/'
[junit4:junit4]   2> 9083 T96 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solr-test-home-21381301548538531/collection1/lib/README' to classloader
[junit4:junit4]   2> 9083 T96 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solr-test-home-21381301548538531/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 9138 T96 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 9187 T96 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 9188 T96 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 9194 T96 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 9700 T96 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 9718 T96 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 9721 T96 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 9731 T96 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9734 T96 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9737 T96 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9738 T96 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9739 T96 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1/data/
[junit4:junit4]   2> 9740 T96 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@682efa99
[junit4:junit4]   2> 9741 T96 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1/data/index/
[junit4:junit4]   2> 9741 T96 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 9743 T96 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 9743 T96 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 9744 T96 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 9744 T96 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 9745 T96 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 9745 T96 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 9745 T96 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 9746 T96 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 9746 T96 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 9747 T96 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 9747 T96 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 9748 T96 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 9748 T96 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 9748 T96 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 9749 T96 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 9749 T96 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 9750 T96 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9750 T96 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9751 T96 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9751 T96 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9751 T96 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9752 T96 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9752 T96 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9753 T96 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 9753 T96 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 9762 T96 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 9763 T96 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 9764 T96 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [THROW.ERROR.ON.ADD.UPDATEPROCESSOR.JS, SOLRCONFIG-HIGHLIGHT.XML, SCHEMA-REQUIRED-FIELDS.XML, SCHEMA-REPLICATION2.XML, TRIVIAL.UPDATEPROCESSOR1.JS, SCHEMA-MINIMAL.XML, BAD-SCHEMA-DUP-DYNAMICFIELD.XML, SOLRCONFIG-CACHING.XML, SOLRCONFIG-REPEATER.XML, BAD-SCHEMA-NONTEXT-ANALYZER.XML, CURRENCY.XML, SOLRCONFIG-MERGEPOLICY.XML, SOLRCONFIG-TLOG.XML, SOLRCONFIG-MASTER.XML, BAD-SCHEMA-UNIQUEKEY-MULTIVALUED.XML, SCHEMA11.XML, SOLRCONFIG-BASIC.XML, DA_COMPOUNDDICTIONARY.TXT, MISSLEADING.EXTENSION.UPDATEPROCESSOR.JS.TXT, SCHEMA-COPYFIELD-TEST.XML, SOLRCONFIG-SLAVE.XML, SOLRCONFIG-PROPINJECT-INDEXDEFAULT.XML, ELEVATE.XML, TRIVIAL.UPDATEPROCESSOR0.JS, BAD-SOLRCONFIG-BOGUS-SCRIPTENGINE-NAME.XML, SCHEMA-IB.XML, SCHEMA-CHARFILTERS.XML, SOLRCONFIG-QUERYSENDER.XML, DA_UTF8.XML, SCHEMA-REPLICATION1.XML, SOLRCONFIG-SNIPPET-PROCESSOR.XML, CONDITIONAL.UPDATEPROCESSOR.JS, MISSING.FUNCTIONS.UPDATEPROCESSOR.JS, SOLRCONFIG-ENABLEPLUGIN.XML, HYPHENATION.DTD, STEMDICT.TXT, SCHEMA-PHRASESUGGEST.XML, HUNSPELL-TEST.AFF, SCHEMA-SNIPPET-TYPE.XML, STOPTYPES-1.TXT, STOPWORDSWRONGENCODING.TXT, SCHEMA-NUMERIC.XML, SOLRCONFIG-TRANSFORMERS.XML, SOLRCONFIG-PROPINJECT.XML, BAD-SCHEMA-NOT-INDEXED-BUT-TF.XML, SOLRCONFIG-SIMPLELOCK.XML, WDFTYPES.TXT, SCHEMA-REVERSED.XML, STOPTYPES-2.TXT, BAD-SCHEMA-CURRENCY-FT-MULTIVALUED.XML, SOLRCONFIG-SPELLCHECKCOMPONENT.XML, BAD-SCHEMA-CURRENCY-MULTIVALUED.XML, SCHEMA-DFR.XML, SOLRCONFIG-PHRASESUGGEST.XML, BAD-SCHEMA-NOT-INDEXED-BUT-POS.XML, KEEP-1.TXT, OPEN-EXCHANGE-RATES.JSON, SCHEMA-SPATIAL.XML, STOPWITHBOM.TXT, SOLRCONFIG-SPELLCHECKER.XML, SCHEMA-BINARYFIELD.XML, SOLRCONFIG-UPDATE-PROCESSOR-CHAINS.XML, BAD-SCHEMA-OMIT-TF-BUT-NOT-POS.XML, SCHEMA-XINCLUDE.XML, BAD-SCHEMA-DUP-FIELDTYPE.XML, SOLRCONFIG-MASTER1.XML, SYNONYMS.TXT, BAD-SCHEMA-CURRENCY-DYNAMIC-MULTIVALUED.XML, SCHEMA_CODEC.XML, SCHEMA.XML, SOLRCONFIG-SOLR-749.XML, SOLRCONFIG-MASTER1-KEEPONEBACKUP.XML, STOP-2.TXT, SOLRCONFIG-FUNCTIONQUERY.XML, SCHEMA-LMDIRICHLET.XML, SOLRCONFIG-TERMINDEX.XML, SOLRCONFIG-ELEVATE.XML, STOPWORDS.TXT, SCHEMA-FOLDING.XML, SCHEMA-STOP-KEEP.XML, BAD-SCHEMA-NOT-INDEXED-BUT-NORMS.XML, SOLRCONFIG-SOLCOREPROPERTIES.XML, STOP-1.TXT, SOLRCONFIG-MASTER2.XML, SCHEMA-SPELLCHECKER.XML, SOLRCONFIG-RESPONSE-LOG-COMPONENT.XML, SOLRCONFIG-LAZYWRITER.XML, SCHEMA-LUCENEMATCHVERSION.XML, BAD-MP-SOLRCONFIG.XML, FRENCHARTICLES.TXT, SCHEMA15.XML, SOLRCONFIG-REQHANDLER.INCL, SCHEMASURROUND.XML, SCHEMA-COLLATEFILTER.XML, SOLRCONFIG-MASTER3.XML, HUNSPELL-TEST.DIC, BAD-SCHEMA-CODEC-GLOBAL-VS-FT-MISMATCH.XML, SOLRCONFIG-XINCLUDE.XML, SOLRCONFIG-DELPOLICY1.XML, SOLRCONFIG-SLAVE1.XML, SCHEMA-SIM.XML, SCHEMA-SNIPPET-FIELD.XML, SCHEMA-COLLATE.XML, STOP-SNOWBALL.TXT, BAD-SCHEMA-SIM-GLOBAL-VS-FT-MISMATCH.XML, PROTWORDS.TXT, SCHEMA-TRIE.XML, SOLRCONFIG_CODEC.XML, BAD-SOLRCONFIG-INVALID-SCRIPTFILE.XML, JASUGGEST.TXT, SCHEMA-TFIDF.XML, SOLRCONFIG-SCRIPT-UPDATEPROCESSOR.XML, SCHEMA-LMJELINEKMERCER.XML, PHRASESUGGEST.TXT, BAD-SOLRCONFIG-MISSING-SCRIPTFILE.XML, SOLRCONFIG-BASIC-LUCENEVERSION31.XML, BAD-SCHEMA-UNIQUEKEY-USES-DEFAULT.XML, OLD_SYNONYMS.TXT, SOLRCONFIG-DELPOLICY2.XML, XSLT, SOLRCONFIG-NATIVELOCK.XML, BAD-SCHEMA-DUP-FIELD.XML, SOLRCONFIG-NOCACHE.XML, SCHEMA-BM25.XML, ADDFIELDS.UPDATEPROCESSOR.JS, SOLRCONFIG-QUERYSENDER-NOQUERY.XML, SOLRCONFIG-ALTDIRECTORY.XML, COMPOUNDDICTIONARY.TXT, SOLRCONFIG-INDEXCONFIG.XML, SOLRCONFIG_PERF.XML, BAD-SCHEMA-ANALYZER-CLASS-AND-NESTED.XML, SCHEMA-NOT-REQUIRED-UNIQUE-KEY.XML, KEEP-2.TXT, BAD-SCHEMA-UNIQUEKEY-IS-COPYFIELD-DEST.XML, SCHEMA12.XML, MAPPING-ISOLATIN1ACCENT.TXT, BAD_SOLRCONFIG.XML, BAD-SCHEMA-EXTERNAL-FILEFIELD.XML]
[junit4:junit4]   2> 9771 T96 oass.SolrIndexSearcher.<init> Opening Searcher@5cd76582 main
[junit4:junit4]   2> 9771 T96 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 9772 T96 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 9772 T96 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 9772 T96 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 9774 T96 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 9788 T98 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5cd76582 main{StandardDirectoryReader(segments_2:3:nrt _0(4.1):C1)}
[junit4:junit4]   2> 9792 T96 oasc.CoreContainer.register replacing core: collection1
[junit4:junit4]   2> 9792 T96 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@3d88eb46
[junit4:junit4]   2> 9797 T96 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=1,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 9797 T96 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 9799 T81 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C15 name=collection1 org.apache.solr.core.SolrCore@3136505b
[junit4:junit4]   2> 9816 T94 C15 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 9827 T94 C15 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@680e6039 lockFactory=org.apache.lucene.store.NativeFSLockFactory@60e5272b; maxCacheMB=0.8359375 maxMergeSizeMB=0.57421875))),segFN=segments_2,generation=2,filenames=[_0.fnm, _0_TestBloomFilteredLucene41Postings_0.blm, _0_MockSep_0.pos, _0_Lucene41_0.pos, _0_TestBloomFilteredLucene41Postings_0.tim, _0_nrm.cfs, _0_TestBloomFilteredLucene41Postings_0.tip, _0_TestBloomFilteredLucene41Postings_0.pos, _0_MockSep_0.tib, _0_Lucene41_0.doc, _0_MockSep_0.skp, _0_nrm.cfe, _0_TestBloomFilteredLucene41Postings_0.doc, _0_MockSep_0.frq, segments_2, _0.fdx, _0_MockSep_0.tii, _0.si, _0_MockSep_0.pyl, _0_MockSep_0.doc, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@680e6039 lockFactory=org.apache.lucene.store.NativeFSLockFactory@60e5272b; maxCacheMB=0.8359375 maxMergeSizeMB=0.57421875))),segFN=segments_3,generation=3,filenames=[_1_MockSep_0.doc, _1_MockSep_0.frq, _1_MockSep_0.pyl, _1_nrm.cfs, _1_TestBloomFilteredLucene41Postings_0.pos, _1_Lucene41_0.doc, _1_MockSep_0.skp, _1_MockSep_0.pos, _1_TestBloomFilteredLucene41Postings_0.tim, _1_MockSep_0.tib, _1_Lucene41_0.tim, _1_Lucene41_0.tip, _1_TestBloomFilteredLucene41Postings_0.doc, _1_TestBloomFilteredLucene41Postings_0.tip, _1.fnm, _1_MockSep_0.tii, _1_nrm.cfe, _1.fdx, _1_Lucene41_0.pos, _1_TestBloomFilteredLucene41Postings_0.blm, _1.fdt, _1.si, segments_3]
[junit4:junit4]   2> 9827 T94 C15 oasc.SolrDeletionPolicy.updateCommits newest commit = 3
[junit4:junit4]   2> 9829 T94 C15 oass.SolrIndexSearcher.<init> Opening Searcher@1de9a4d0 main
[junit4:junit4]   2> 9830 T94 C15 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 9830 T94 C15 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 9832 T98 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1de9a4d0 main{StandardDirectoryReader(segments_3:7:nrt _1(4.1):C1)}
[junit4:junit4]   2> 9832 T94 C15 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {add=[42],commit=} 0 18
[junit4:junit4]   2> 9833 T81 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9843 T95 C15 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=teststop:terma&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 9844 T81 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9857 T96 C15 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=teststop:stopworda&version=2} hits=1 status=0 QTime=2 
[junit4:junit4]   2> 9858 T81 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9868 T94 C15 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=teststop:stopwordb&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 9869 T81 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9878 T95 C15 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=teststop:stopwordc&version=2} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 9924 T81 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1932761421
[junit4:junit4]   2> 9924 T81 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@3136505b
[junit4:junit4]   2> 9933 T81 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=1,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 9934 T81 oasc.SolrCore.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 9934 T81 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 9934 T81 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 9936 T81 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 9944 T81 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 10045 T81 oejut.QueuedThreadPool.doStop WARNING 4 threads could not be stopped
[junit4:junit4]   2> 10094 T81 oasu.AbstractSolrTestCase.tearDown ####TEARDOWN_START testStopwordsAfterCoreReload
[junit4:junit4]   2> 10095 T81 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=75451581
[junit4:junit4]   2> 10095 T81 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@41870e99
[junit4:junit4]   2> 10097 T81 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 10097 T81 oasc.SolrCore.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 10098 T81 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 10098 T81 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 10098 T81 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 10112 T80 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 4 leaked thread(s).
[junit4:junit4]   2> 30240 T80 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 4 threads leaked from SUITE scope at org.apache.solr.AnalysisAfterCoreReloadTest: 
[junit4:junit4]   2> 	   1) Thread[id=92, name=qtp1156597185-92 Acceptor3 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   2) Thread[id=91, name=qtp1156597185-91 Acceptor2 SelectChannelConnector@0.0.0.0:29439, state=RUNNABLE, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   3) Thread[id=89, name=qtp1156597185-89 Acceptor0 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   4) Thread[id=90, name=qtp1156597185-90 Acceptor1 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 30241 T80 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
[junit4:junit4]   2> 	   1) Thread[id=92, name=qtp1156597185-92 Acceptor3 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]   2> 	   2) Thread[id=91, name=qtp1156597185-91 Acceptor2 SelectChannelConnector@0.0.0.0:29439, state=RUNNABLE, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]   2> 	   3) Thread[id=89, name=qtp1156597185-89 Acceptor0 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]   2> 	   4) Thread[id=90, name=qtp1156597185-90 Acceptor1 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]   2> 33284 T80 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
[junit4:junit4]   2> 	   1) Thread[id=92, name=qtp1156597185-92 Acceptor3 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   2) Thread[id=91, name=qtp1156597185-91 Acceptor2 SelectChannelConnector@0.0.0.0:29439, state=RUNNABLE, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   3) Thread[id=89, name=qtp1156597185-89 Acceptor0 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   4) Thread[id=90, name=qtp1156597185-90 Acceptor1 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene41: {timestamp=Lucene41(blocksize=128), range_facet_l=Lucene41(blocksize=128), id=PostingsFormat(name=MockSep), range_facet_sl=PostingsFormat(name=MockSep), range_facet_si=Lucene41(blocksize=128), teststop=Lucene41(blocksize=128), multiDefault=Lucene41(blocksize=128), intDefault=PostingsFormat(name=TestBloomFilteredLucene41Postings)}, sim=DefaultSimilarity, locale=ar_BH, timezone=CAT
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=5,free=77474648,total=130023424
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestLuceneMatchVersion, UniqFieldsUpdateProcessorFactoryTest, AutoCommitTest, MoreLikeThisHandlerTest, TestCoreContainer, FastVectorHighlighterTest, TestGroupingSearch, QueryElevationComponentTest, AnalysisAfterCoreReloadTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=AnalysisAfterCoreReloadTest -Dtests.seed=EFEC412AFD1FA1E8 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar_BH -Dtests.timezone=CAT -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR   0.00s J0 | AnalysisAfterCoreReloadTest (suite) <<<
[junit4:junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.AnalysisAfterCoreReloadTest: 
[junit4:junit4]    >    1) Thread[id=92, name=qtp1156597185-92 Acceptor3 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=91, name=qtp1156597185-91 Acceptor2 SelectChannelConnector@0.0.0.0:29439, state=RUNNABLE, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=89, name=qtp1156597185-89 Acceptor0 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=90, name=qtp1156597185-90 Acceptor1 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([EFEC412AFD1FA1E8]:0)
[junit4:junit4]    > Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4]    >    1) Thread[id=92, name=qtp1156597185-92 Acceptor3 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=91, name=qtp1156597185-91 Acceptor2 SelectChannelConnector@0.0.0.0:29439, state=RUNNABLE, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=89, name=qtp1156597185-89 Acceptor0 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=90, name=qtp1156597185-90 Acceptor1 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([EFEC412AFD1FA1E8]:0)
[junit4:junit4] Completed on J0 in 28.40s, 1 test, 2 errors <<< FAILURES!

[...truncated 152 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:335: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/module-build.xml:61: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:1163: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:827: There were test failures: 233 suites, 970 tests, 4 suite-level errors, 826 ignored

Total time: 28 minutes 8 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure

[JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 614 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.

Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/614/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.TestDistributedGrouping

Error Message:
36 threads leaked from SUITE scope at org.apache.solr.TestDistributedGrouping:     1) Thread[id=91, name=qtp1758179843-91 Acceptor3 SelectChannelConnector@0.0.0.0:43789, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=53, name=qtp702834017-53 Acceptor0 SelectChannelConnector@0.0.0.0:16366, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=88, name=qtp1758179843-88 Acceptor0 SelectChannelConnector@0.0.0.0:43789, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=89, name=qtp1758179843-89 Acceptor1 SelectChannelConnector@0.0.0.0:43789, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=145, name=qtp979199720-145 Acceptor3 SelectChannelConnector@0.0.0.0:23576, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=54, name=qtp702834017-54 Acceptor1 SelectChannelConnector@0.0.0.0:16366, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=76, name=qtp778771945-76 Acceptor2 SelectChannelConnector@0.0.0.0:54577, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=143, name=qtp979199720-143 Acceptor1 SelectChannelConnector@0.0.0.0:23576, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=169, name=qtp1282480504-169 Acceptor1 SelectChannelConnector@0.0.0.0:62606, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=102, name=qtp284490634-102 Acceptor0 SelectChannelConnector@0.0.0.0:35039, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=104, name=qtp284490634-104 Acceptor2 SelectChannelConnector@0.0.0.0:35039, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=142, name=qtp979199720-142 Acceptor0 SelectChannelConnector@0.0.0.0:23576, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=157, name=qtp1165625747-157 Acceptor3 SelectChannelConnector@0.0.0.0:41277, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=156, name=qtp1165625747-156 Acceptor2 SelectChannelConnector@0.0.0.0:41277, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=155, name=qtp1165625747-155 Acceptor1 SelectChannelConnector@0.0.0.0:41277, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=75, name=qtp778771945-75 Acceptor1 SelectChannelConnector@0.0.0.0:54577, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=90, name=qtp1758179843-90 Acceptor2 SelectChannelConnector@0.0.0.0:43789, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=171, name=qtp1282480504-171 Acceptor3 SelectChannelConnector@0.0.0.0:62606, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=154, name=qtp1165625747-154 Acceptor0 SelectChannelConnector@0.0.0.0:41277, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=144, name=qtp979199720-144 Acceptor2 SelectChannelConnector@0.0.0.0:23576, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   21) Thread[id=130, name=qtp1864401185-130 Acceptor2 SelectChannelConnector@0.0.0.0:22609, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   22) Thread[id=77, name=qtp778771945-77 Acceptor3 SelectChannelConnector@0.0.0.0:54577, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   23) Thread[id=129, name=qtp1864401185-129 Acceptor1 SelectChannelConnector@0.0.0.0:22609, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   24) Thread[id=105, name=qtp284490634-105 Acceptor3 SelectChannelConnector@0.0.0.0:35039, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   25) Thread[id=168, name=qtp1282480504-168 Acceptor0 SelectChannelConnector@0.0.0.0:62606, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   26) Thread[id=42, name=qtp382843504-42 Acceptor3 SelectChannelConnector@0.0.0.0:53909, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   27) Thread[id=55, name=qtp702834017-55 Acceptor2 SelectChannelConnector@0.0.0.0:16366, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   28) Thread[id=39, name=qtp382843504-39 Acceptor0 SelectChannelConnector@0.0.0.0:53909, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   29) Thread[id=74, name=qtp778771945-74 Acceptor0 SelectChannelConnector@0.0.0.0:54577, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   30) Thread[id=56, name=qtp702834017-56 Acceptor3 SelectChannelConnector@0.0.0.0:16366, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   31) Thread[id=170, name=qtp1282480504-170 Acceptor2 SelectChannelConnector@0.0.0.0:62606, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   32) Thread[id=128, name=qtp1864401185-128 Acceptor0 SelectChannelConnector@0.0.0.0:22609, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   33) Thread[id=131, name=qtp1864401185-131 Acceptor3 SelectChannelConnector@0.0.0.0:22609, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   34) Thread[id=40, name=qtp382843504-40 Acceptor1 SelectChannelConnector@0.0.0.0:53909, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   35) Thread[id=103, name=qtp284490634-103 Acceptor1 SelectChannelConnector@0.0.0.0:35039, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   36) Thread[id=41, name=qtp382843504-41 Acceptor2 SelectChannelConnector@0.0.0.0:53909, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 36 threads leaked from SUITE scope at org.apache.solr.TestDistributedGrouping: 
   1) Thread[id=91, name=qtp1758179843-91 Acceptor3 SelectChannelConnector@0.0.0.0:43789, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=53, name=qtp702834017-53 Acceptor0 SelectChannelConnector@0.0.0.0:16366, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=88, name=qtp1758179843-88 Acceptor0 SelectChannelConnector@0.0.0.0:43789, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=89, name=qtp1758179843-89 Acceptor1 SelectChannelConnector@0.0.0.0:43789, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=145, name=qtp979199720-145 Acceptor3 SelectChannelConnector@0.0.0.0:23576, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=54, name=qtp702834017-54 Acceptor1 SelectChannelConnector@0.0.0.0:16366, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=76, name=qtp778771945-76 Acceptor2 SelectChannelConnector@0.0.0.0:54577, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=143, name=qtp979199720-143 Acceptor1 SelectChannelConnector@0.0.0.0:23576, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=169, name=qtp1282480504-169 Acceptor1 SelectChannelConnector@0.0.0.0:62606, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=102, name=qtp284490634-102 Acceptor0 SelectChannelConnector@0.0.0.0:35039, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=104, name=qtp284490634-104 Acceptor2 SelectChannelConnector@0.0.0.0:35039, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=142, name=qtp979199720-142 Acceptor0 SelectChannelConnector@0.0.0.0:23576, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=157, name=qtp1165625747-157 Acceptor3 SelectChannelConnector@0.0.0.0:41277, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=156, name=qtp1165625747-156 Acceptor2 SelectChannelConnector@0.0.0.0:41277, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=155, name=qtp1165625747-155 Acceptor1 SelectChannelConnector@0.0.0.0:41277, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=75, name=qtp778771945-75 Acceptor1 SelectChannelConnector@0.0.0.0:54577, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=90, name=qtp1758179843-90 Acceptor2 SelectChannelConnector@0.0.0.0:43789, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=171, name=qtp1282480504-171 Acceptor3 SelectChannelConnector@0.0.0.0:62606, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=154, name=qtp1165625747-154 Acceptor0 SelectChannelConnector@0.0.0.0:41277, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=144, name=qtp979199720-144 Acceptor2 SelectChannelConnector@0.0.0.0:23576, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  21) Thread[id=130, name=qtp1864401185-130 Acceptor2 SelectChannelConnector@0.0.0.0:22609, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  22) Thread[id=77, name=qtp778771945-77 Acceptor3 SelectChannelConnector@0.0.0.0:54577, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  23) Thread[id=129, name=qtp1864401185-129 Acceptor1 SelectChannelConnector@0.0.0.0:22609, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  24) Thread[id=105, name=qtp284490634-105 Acceptor3 SelectChannelConnector@0.0.0.0:35039, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  25) Thread[id=168, name=qtp1282480504-168 Acceptor0 SelectChannelConnector@0.0.0.0:62606, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  26) Thread[id=42, name=qtp382843504-42 Acceptor3 SelectChannelConnector@0.0.0.0:53909, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  27) Thread[id=55, name=qtp702834017-55 Acceptor2 SelectChannelConnector@0.0.0.0:16366, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  28) Thread[id=39, name=qtp382843504-39 Acceptor0 SelectChannelConnector@0.0.0.0:53909, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  29) Thread[id=74, name=qtp778771945-74 Acceptor0 SelectChannelConnector@0.0.0.0:54577, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  30) Thread[id=56, name=qtp702834017-56 Acceptor3 SelectChannelConnector@0.0.0.0:16366, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  31) Thread[id=170, name=qtp1282480504-170 Acceptor2 SelectChannelConnector@0.0.0.0:62606, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  32) Thread[id=128, name=qtp1864401185-128 Acceptor0 SelectChannelConnector@0.0.0.0:22609, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  33) Thread[id=131, name=qtp1864401185-131 Acceptor3 SelectChannelConnector@0.0.0.0:22609, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  34) Thread[id=40, name=qtp382843504-40 Acceptor1 SelectChannelConnector@0.0.0.0:53909, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  35) Thread[id=103, name=qtp284490634-103 Acceptor1 SelectChannelConnector@0.0.0.0:35039, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  36) Thread[id=41, name=qtp382843504-41 Acceptor2 SelectChannelConnector@0.0.0.0:53909, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([E10CCE4D9DEDE79A]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.TestDistributedGrouping

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=91, name=qtp1758179843-91 Acceptor3 SelectChannelConnector@0.0.0.0:43789, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=53, name=qtp702834017-53 Acceptor0 SelectChannelConnector@0.0.0.0:16366, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=88, name=qtp1758179843-88 Acceptor0 SelectChannelConnector@0.0.0.0:43789, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=89, name=qtp1758179843-89 Acceptor1 SelectChannelConnector@0.0.0.0:43789, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=145, name=qtp979199720-145 Acceptor3 SelectChannelConnector@0.0.0.0:23576, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=54, name=qtp702834017-54 Acceptor1 SelectChannelConnector@0.0.0.0:16366, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=76, name=qtp778771945-76 Acceptor2 SelectChannelConnector@0.0.0.0:54577, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=143, name=qtp979199720-143 Acceptor1 SelectChannelConnector@0.0.0.0:23576, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=169, name=qtp1282480504-169 Acceptor1 SelectChannelConnector@0.0.0.0:62606, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=102, name=qtp284490634-102 Acceptor0 SelectChannelConnector@0.0.0.0:35039, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=104, name=qtp284490634-104 Acceptor2 SelectChannelConnector@0.0.0.0:35039, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=142, name=qtp979199720-142 Acceptor0 SelectChannelConnector@0.0.0.0:23576, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=157, name=qtp1165625747-157 Acceptor3 SelectChannelConnector@0.0.0.0:41277, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=155, name=qtp1165625747-155 Acceptor1 SelectChannelConnector@0.0.0.0:41277, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=156, name=qtp1165625747-156 Acceptor2 SelectChannelConnector@0.0.0.0:41277, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=75, name=qtp778771945-75 Acceptor1 SelectChannelConnector@0.0.0.0:54577, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=90, name=qtp1758179843-90 Acceptor2 SelectChannelConnector@0.0.0.0:43789, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=144, name=qtp979199720-144 Acceptor2 SelectChannelConnector@0.0.0.0:23576, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=154, name=qtp1165625747-154 Acceptor0 SelectChannelConnector@0.0.0.0:41277, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=171, name=qtp1282480504-171 Acceptor3 SelectChannelConnector@0.0.0.0:62606, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   21) Thread[id=77, name=qtp778771945-77 Acceptor3 SelectChannelConnector@0.0.0.0:54577, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   22) Thread[id=130, name=qtp1864401185-130 Acceptor2 SelectChannelConnector@0.0.0.0:22609, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   23) Thread[id=105, name=qtp284490634-105 Acceptor3 SelectChannelConnector@0.0.0.0:35039, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   24) Thread[id=129, name=qtp1864401185-129 Acceptor1 SelectChannelConnector@0.0.0.0:22609, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   25) Thread[id=168, name=qtp1282480504-168 Acceptor0 SelectChannelConnector@0.0.0.0:62606, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   26) Thread[id=42, name=qtp382843504-42 Acceptor3 SelectChannelConnector@0.0.0.0:53909, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   27) Thread[id=55, name=qtp702834017-55 Acceptor2 SelectChannelConnector@0.0.0.0:16366, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   28) Thread[id=39, name=qtp382843504-39 Acceptor0 SelectChannelConnector@0.0.0.0:53909, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   29) Thread[id=74, name=qtp778771945-74 Acceptor0 SelectChannelConnector@0.0.0.0:54577, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   30) Thread[id=56, name=qtp702834017-56 Acceptor3 SelectChannelConnector@0.0.0.0:16366, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   31) Thread[id=170, name=qtp1282480504-170 Acceptor2 SelectChannelConnector@0.0.0.0:62606, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   32) Thread[id=128, name=qtp1864401185-128 Acceptor0 SelectChannelConnector@0.0.0.0:22609, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   33) Thread[id=131, name=qtp1864401185-131 Acceptor3 SelectChannelConnector@0.0.0.0:22609, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   34) Thread[id=40, name=qtp382843504-40 Acceptor1 SelectChannelConnector@0.0.0.0:53909, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   35) Thread[id=103, name=qtp284490634-103 Acceptor1 SelectChannelConnector@0.0.0.0:35039, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   36) Thread[id=41, name=qtp382843504-41 Acceptor2 SelectChannelConnector@0.0.0.0:53909, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=91, name=qtp1758179843-91 Acceptor3 SelectChannelConnector@0.0.0.0:43789, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=53, name=qtp702834017-53 Acceptor0 SelectChannelConnector@0.0.0.0:16366, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=88, name=qtp1758179843-88 Acceptor0 SelectChannelConnector@0.0.0.0:43789, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=89, name=qtp1758179843-89 Acceptor1 SelectChannelConnector@0.0.0.0:43789, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=145, name=qtp979199720-145 Acceptor3 SelectChannelConnector@0.0.0.0:23576, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=54, name=qtp702834017-54 Acceptor1 SelectChannelConnector@0.0.0.0:16366, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=76, name=qtp778771945-76 Acceptor2 SelectChannelConnector@0.0.0.0:54577, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=143, name=qtp979199720-143 Acceptor1 SelectChannelConnector@0.0.0.0:23576, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=169, name=qtp1282480504-169 Acceptor1 SelectChannelConnector@0.0.0.0:62606, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=102, name=qtp284490634-102 Acceptor0 SelectChannelConnector@0.0.0.0:35039, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=104, name=qtp284490634-104 Acceptor2 SelectChannelConnector@0.0.0.0:35039, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=142, name=qtp979199720-142 Acceptor0 SelectChannelConnector@0.0.0.0:23576, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=157, name=qtp1165625747-157 Acceptor3 SelectChannelConnector@0.0.0.0:41277, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=155, name=qtp1165625747-155 Acceptor1 SelectChannelConnector@0.0.0.0:41277, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=156, name=qtp1165625747-156 Acceptor2 SelectChannelConnector@0.0.0.0:41277, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=75, name=qtp778771945-75 Acceptor1 SelectChannelConnector@0.0.0.0:54577, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=90, name=qtp1758179843-90 Acceptor2 SelectChannelConnector@0.0.0.0:43789, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=144, name=qtp979199720-144 Acceptor2 SelectChannelConnector@0.0.0.0:23576, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=154, name=qtp1165625747-154 Acceptor0 SelectChannelConnector@0.0.0.0:41277, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=171, name=qtp1282480504-171 Acceptor3 SelectChannelConnector@0.0.0.0:62606, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  21) Thread[id=77, name=qtp778771945-77 Acceptor3 SelectChannelConnector@0.0.0.0:54577, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  22) Thread[id=130, name=qtp1864401185-130 Acceptor2 SelectChannelConnector@0.0.0.0:22609, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  23) Thread[id=105, name=qtp284490634-105 Acceptor3 SelectChannelConnector@0.0.0.0:35039, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  24) Thread[id=129, name=qtp1864401185-129 Acceptor1 SelectChannelConnector@0.0.0.0:22609, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  25) Thread[id=168, name=qtp1282480504-168 Acceptor0 SelectChannelConnector@0.0.0.0:62606, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  26) Thread[id=42, name=qtp382843504-42 Acceptor3 SelectChannelConnector@0.0.0.0:53909, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  27) Thread[id=55, name=qtp702834017-55 Acceptor2 SelectChannelConnector@0.0.0.0:16366, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  28) Thread[id=39, name=qtp382843504-39 Acceptor0 SelectChannelConnector@0.0.0.0:53909, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  29) Thread[id=74, name=qtp778771945-74 Acceptor0 SelectChannelConnector@0.0.0.0:54577, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  30) Thread[id=56, name=qtp702834017-56 Acceptor3 SelectChannelConnector@0.0.0.0:16366, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  31) Thread[id=170, name=qtp1282480504-170 Acceptor2 SelectChannelConnector@0.0.0.0:62606, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  32) Thread[id=128, name=qtp1864401185-128 Acceptor0 SelectChannelConnector@0.0.0.0:22609, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  33) Thread[id=131, name=qtp1864401185-131 Acceptor3 SelectChannelConnector@0.0.0.0:22609, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  34) Thread[id=40, name=qtp382843504-40 Acceptor1 SelectChannelConnector@0.0.0.0:53909, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  35) Thread[id=103, name=qtp284490634-103 Acceptor1 SelectChannelConnector@0.0.0.0:35039, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  36) Thread[id=41, name=qtp382843504-41 Acceptor2 SelectChannelConnector@0.0.0.0:53909, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([E10CCE4D9DEDE79A]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.component.DistributedSpellCheckComponentTest

Error Message:
36 threads leaked from SUITE scope at org.apache.solr.handler.component.DistributedSpellCheckComponentTest:     1) Thread[id=373, name=qtp1166599758-373 Acceptor1 SelectChannelConnector@0.0.0.0:16534, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=328, name=qtp869068009-328 Acceptor0 SelectChannelConnector@0.0.0.0:36572, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=420, name=qtp2066373459-420 Acceptor1 SelectChannelConnector@0.0.0.0:32002, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=421, name=qtp2066373459-421 Acceptor2 SelectChannelConnector@0.0.0.0:32002, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=360, name=qtp1359724161-360 Acceptor2 SelectChannelConnector@0.0.0.0:10890, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=406, name=qtp279283093-406 Acceptor0 SelectChannelConnector@0.0.0.0:35826, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=449, name=qtp1011276511-449 Acceptor2 SelectChannelConnector@0.0.0.0:56217, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=342, name=qtp1562851016-342 Acceptor0 SelectChannelConnector@0.0.0.0:56921, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=344, name=qtp1562851016-344 Acceptor2 SelectChannelConnector@0.0.0.0:56921, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=388, name=qtp1396210494-388 Acceptor2 SelectChannelConnector@0.0.0.0:64726, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=447, name=qtp1011276511-447 Acceptor0 SelectChannelConnector@0.0.0.0:56217, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=345, name=qtp1562851016-345 Acceptor3 SelectChannelConnector@0.0.0.0:56921, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=434, name=qtp1951812991-434 Acceptor1 SelectChannelConnector@0.0.0.0:46423, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=372, name=qtp1166599758-372 Acceptor0 SelectChannelConnector@0.0.0.0:16534, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=433, name=qtp1951812991-433 Acceptor0 SelectChannelConnector@0.0.0.0:46423, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=330, name=qtp869068009-330 Acceptor2 SelectChannelConnector@0.0.0.0:36572, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=359, name=qtp1359724161-359 Acceptor1 SelectChannelConnector@0.0.0.0:10890, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=358, name=qtp1359724161-358 Acceptor0 SelectChannelConnector@0.0.0.0:10890, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=361, name=qtp1359724161-361 Acceptor3 SelectChannelConnector@0.0.0.0:10890, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=422, name=qtp2066373459-422 Acceptor3 SelectChannelConnector@0.0.0.0:32002, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   21) Thread[id=343, name=qtp1562851016-343 Acceptor1 SelectChannelConnector@0.0.0.0:56921, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   22) Thread[id=450, name=qtp1011276511-450 Acceptor3 SelectChannelConnector@0.0.0.0:56217, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   23) Thread[id=419, name=qtp2066373459-419 Acceptor0 SelectChannelConnector@0.0.0.0:32002, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   24) Thread[id=448, name=qtp1011276511-448 Acceptor1 SelectChannelConnector@0.0.0.0:56217, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   25) Thread[id=329, name=qtp869068009-329 Acceptor1 SelectChannelConnector@0.0.0.0:36572, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   26) Thread[id=331, name=qtp869068009-331 Acceptor3 SelectChannelConnector@0.0.0.0:36572, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   27) Thread[id=389, name=qtp1396210494-389 Acceptor3 SelectChannelConnector@0.0.0.0:64726, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   28) Thread[id=386, name=qtp1396210494-386 Acceptor0 SelectChannelConnector@0.0.0.0:64726, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   29) Thread[id=407, name=qtp279283093-407 Acceptor1 SelectChannelConnector@0.0.0.0:35826, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   30) Thread[id=409, name=qtp279283093-409 Acceptor3 SelectChannelConnector@0.0.0.0:35826, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   31) Thread[id=375, name=qtp1166599758-375 Acceptor3 SelectChannelConnector@0.0.0.0:16534, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   32) Thread[id=408, name=qtp279283093-408 Acceptor2 SelectChannelConnector@0.0.0.0:35826, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   33) Thread[id=436, name=qtp1951812991-436 Acceptor3 SelectChannelConnector@0.0.0.0:46423, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   34) Thread[id=374, name=qtp1166599758-374 Acceptor2 SelectChannelConnector@0.0.0.0:16534, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   35) Thread[id=387, name=qtp1396210494-387 Acceptor1 SelectChannelConnector@0.0.0.0:64726, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   36) Thread[id=435, name=qtp1951812991-435 Acceptor2 SelectChannelConnector@0.0.0.0:46423, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 36 threads leaked from SUITE scope at org.apache.solr.handler.component.DistributedSpellCheckComponentTest: 
   1) Thread[id=373, name=qtp1166599758-373 Acceptor1 SelectChannelConnector@0.0.0.0:16534, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=328, name=qtp869068009-328 Acceptor0 SelectChannelConnector@0.0.0.0:36572, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=420, name=qtp2066373459-420 Acceptor1 SelectChannelConnector@0.0.0.0:32002, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=421, name=qtp2066373459-421 Acceptor2 SelectChannelConnector@0.0.0.0:32002, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=360, name=qtp1359724161-360 Acceptor2 SelectChannelConnector@0.0.0.0:10890, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=406, name=qtp279283093-406 Acceptor0 SelectChannelConnector@0.0.0.0:35826, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=449, name=qtp1011276511-449 Acceptor2 SelectChannelConnector@0.0.0.0:56217, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=342, name=qtp1562851016-342 Acceptor0 SelectChannelConnector@0.0.0.0:56921, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=344, name=qtp1562851016-344 Acceptor2 SelectChannelConnector@0.0.0.0:56921, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=388, name=qtp1396210494-388 Acceptor2 SelectChannelConnector@0.0.0.0:64726, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=447, name=qtp1011276511-447 Acceptor0 SelectChannelConnector@0.0.0.0:56217, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=345, name=qtp1562851016-345 Acceptor3 SelectChannelConnector@0.0.0.0:56921, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=434, name=qtp1951812991-434 Acceptor1 SelectChannelConnector@0.0.0.0:46423, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=372, name=qtp1166599758-372 Acceptor0 SelectChannelConnector@0.0.0.0:16534, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=433, name=qtp1951812991-433 Acceptor0 SelectChannelConnector@0.0.0.0:46423, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=330, name=qtp869068009-330 Acceptor2 SelectChannelConnector@0.0.0.0:36572, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=359, name=qtp1359724161-359 Acceptor1 SelectChannelConnector@0.0.0.0:10890, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=358, name=qtp1359724161-358 Acceptor0 SelectChannelConnector@0.0.0.0:10890, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=361, name=qtp1359724161-361 Acceptor3 SelectChannelConnector@0.0.0.0:10890, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=422, name=qtp2066373459-422 Acceptor3 SelectChannelConnector@0.0.0.0:32002, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  21) Thread[id=343, name=qtp1562851016-343 Acceptor1 SelectChannelConnector@0.0.0.0:56921, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  22) Thread[id=450, name=qtp1011276511-450 Acceptor3 SelectChannelConnector@0.0.0.0:56217, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  23) Thread[id=419, name=qtp2066373459-419 Acceptor0 SelectChannelConnector@0.0.0.0:32002, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  24) Thread[id=448, name=qtp1011276511-448 Acceptor1 SelectChannelConnector@0.0.0.0:56217, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  25) Thread[id=329, name=qtp869068009-329 Acceptor1 SelectChannelConnector@0.0.0.0:36572, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  26) Thread[id=331, name=qtp869068009-331 Acceptor3 SelectChannelConnector@0.0.0.0:36572, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  27) Thread[id=389, name=qtp1396210494-389 Acceptor3 SelectChannelConnector@0.0.0.0:64726, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  28) Thread[id=386, name=qtp1396210494-386 Acceptor0 SelectChannelConnector@0.0.0.0:64726, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  29) Thread[id=407, name=qtp279283093-407 Acceptor1 SelectChannelConnector@0.0.0.0:35826, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  30) Thread[id=409, name=qtp279283093-409 Acceptor3 SelectChannelConnector@0.0.0.0:35826, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  31) Thread[id=375, name=qtp1166599758-375 Acceptor3 SelectChannelConnector@0.0.0.0:16534, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  32) Thread[id=408, name=qtp279283093-408 Acceptor2 SelectChannelConnector@0.0.0.0:35826, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  33) Thread[id=436, name=qtp1951812991-436 Acceptor3 SelectChannelConnector@0.0.0.0:46423, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  34) Thread[id=374, name=qtp1166599758-374 Acceptor2 SelectChannelConnector@0.0.0.0:16534, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  35) Thread[id=387, name=qtp1396210494-387 Acceptor1 SelectChannelConnector@0.0.0.0:64726, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  36) Thread[id=435, name=qtp1951812991-435 Acceptor2 SelectChannelConnector@0.0.0.0:46423, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([E10CCE4D9DEDE79A]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.component.DistributedSpellCheckComponentTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=373, name=qtp1166599758-373 Acceptor1 SelectChannelConnector@0.0.0.0:16534, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=328, name=qtp869068009-328 Acceptor0 SelectChannelConnector@0.0.0.0:36572, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=420, name=qtp2066373459-420 Acceptor1 SelectChannelConnector@0.0.0.0:32002, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=421, name=qtp2066373459-421 Acceptor2 SelectChannelConnector@0.0.0.0:32002, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=360, name=qtp1359724161-360 Acceptor2 SelectChannelConnector@0.0.0.0:10890, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=406, name=qtp279283093-406 Acceptor0 SelectChannelConnector@0.0.0.0:35826, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=449, name=qtp1011276511-449 Acceptor2 SelectChannelConnector@0.0.0.0:56217, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=342, name=qtp1562851016-342 Acceptor0 SelectChannelConnector@0.0.0.0:56921, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=344, name=qtp1562851016-344 Acceptor2 SelectChannelConnector@0.0.0.0:56921, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=388, name=qtp1396210494-388 Acceptor2 SelectChannelConnector@0.0.0.0:64726, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=447, name=qtp1011276511-447 Acceptor0 SelectChannelConnector@0.0.0.0:56217, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=345, name=qtp1562851016-345 Acceptor3 SelectChannelConnector@0.0.0.0:56921, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=372, name=qtp1166599758-372 Acceptor0 SelectChannelConnector@0.0.0.0:16534, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=434, name=qtp1951812991-434 Acceptor1 SelectChannelConnector@0.0.0.0:46423, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=433, name=qtp1951812991-433 Acceptor0 SelectChannelConnector@0.0.0.0:46423, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=330, name=qtp869068009-330 Acceptor2 SelectChannelConnector@0.0.0.0:36572, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=359, name=qtp1359724161-359 Acceptor1 SelectChannelConnector@0.0.0.0:10890, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=358, name=qtp1359724161-358 Acceptor0 SelectChannelConnector@0.0.0.0:10890, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=361, name=qtp1359724161-361 Acceptor3 SelectChannelConnector@0.0.0.0:10890, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=422, name=qtp2066373459-422 Acceptor3 SelectChannelConnector@0.0.0.0:32002, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   21) Thread[id=343, name=qtp1562851016-343 Acceptor1 SelectChannelConnector@0.0.0.0:56921, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   22) Thread[id=419, name=qtp2066373459-419 Acceptor0 SelectChannelConnector@0.0.0.0:32002, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   23) Thread[id=450, name=qtp1011276511-450 Acceptor3 SelectChannelConnector@0.0.0.0:56217, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   24) Thread[id=448, name=qtp1011276511-448 Acceptor1 SelectChannelConnector@0.0.0.0:56217, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   25) Thread[id=329, name=qtp869068009-329 Acceptor1 SelectChannelConnector@0.0.0.0:36572, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   26) Thread[id=331, name=qtp869068009-331 Acceptor3 SelectChannelConnector@0.0.0.0:36572, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   27) Thread[id=389, name=qtp1396210494-389 Acceptor3 SelectChannelConnector@0.0.0.0:64726, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   28) Thread[id=386, name=qtp1396210494-386 Acceptor0 SelectChannelConnector@0.0.0.0:64726, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   29) Thread[id=407, name=qtp279283093-407 Acceptor1 SelectChannelConnector@0.0.0.0:35826, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   30) Thread[id=409, name=qtp279283093-409 Acceptor3 SelectChannelConnector@0.0.0.0:35826, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   31) Thread[id=375, name=qtp1166599758-375 Acceptor3 SelectChannelConnector@0.0.0.0:16534, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   32) Thread[id=408, name=qtp279283093-408 Acceptor2 SelectChannelConnector@0.0.0.0:35826, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   33) Thread[id=436, name=qtp1951812991-436 Acceptor3 SelectChannelConnector@0.0.0.0:46423, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   34) Thread[id=374, name=qtp1166599758-374 Acceptor2 SelectChannelConnector@0.0.0.0:16534, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   35) Thread[id=387, name=qtp1396210494-387 Acceptor1 SelectChannelConnector@0.0.0.0:64726, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   36) Thread[id=435, name=qtp1951812991-435 Acceptor2 SelectChannelConnector@0.0.0.0:46423, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=373, name=qtp1166599758-373 Acceptor1 SelectChannelConnector@0.0.0.0:16534, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=328, name=qtp869068009-328 Acceptor0 SelectChannelConnector@0.0.0.0:36572, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=420, name=qtp2066373459-420 Acceptor1 SelectChannelConnector@0.0.0.0:32002, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=421, name=qtp2066373459-421 Acceptor2 SelectChannelConnector@0.0.0.0:32002, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=360, name=qtp1359724161-360 Acceptor2 SelectChannelConnector@0.0.0.0:10890, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=406, name=qtp279283093-406 Acceptor0 SelectChannelConnector@0.0.0.0:35826, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=449, name=qtp1011276511-449 Acceptor2 SelectChannelConnector@0.0.0.0:56217, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=342, name=qtp1562851016-342 Acceptor0 SelectChannelConnector@0.0.0.0:56921, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=344, name=qtp1562851016-344 Acceptor2 SelectChannelConnector@0.0.0.0:56921, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=388, name=qtp1396210494-388 Acceptor2 SelectChannelConnector@0.0.0.0:64726, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=447, name=qtp1011276511-447 Acceptor0 SelectChannelConnector@0.0.0.0:56217, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=345, name=qtp1562851016-345 Acceptor3 SelectChannelConnector@0.0.0.0:56921, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=372, name=qtp1166599758-372 Acceptor0 SelectChannelConnector@0.0.0.0:16534, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=434, name=qtp1951812991-434 Acceptor1 SelectChannelConnector@0.0.0.0:46423, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=433, name=qtp1951812991-433 Acceptor0 SelectChannelConnector@0.0.0.0:46423, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=330, name=qtp869068009-330 Acceptor2 SelectChannelConnector@0.0.0.0:36572, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=359, name=qtp1359724161-359 Acceptor1 SelectChannelConnector@0.0.0.0:10890, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=358, name=qtp1359724161-358 Acceptor0 SelectChannelConnector@0.0.0.0:10890, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=361, name=qtp1359724161-361 Acceptor3 SelectChannelConnector@0.0.0.0:10890, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=422, name=qtp2066373459-422 Acceptor3 SelectChannelConnector@0.0.0.0:32002, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  21) Thread[id=343, name=qtp1562851016-343 Acceptor1 SelectChannelConnector@0.0.0.0:56921, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  22) Thread[id=419, name=qtp2066373459-419 Acceptor0 SelectChannelConnector@0.0.0.0:32002, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  23) Thread[id=450, name=qtp1011276511-450 Acceptor3 SelectChannelConnector@0.0.0.0:56217, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  24) Thread[id=448, name=qtp1011276511-448 Acceptor1 SelectChannelConnector@0.0.0.0:56217, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  25) Thread[id=329, name=qtp869068009-329 Acceptor1 SelectChannelConnector@0.0.0.0:36572, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native M

[...truncated too long message...]

ector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   27) Thread[id=389, name=qtp1396210494-389 Acceptor3 SelectChannelConnector@0.0.0.0:64726, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   28) Thread[id=386, name=qtp1396210494-386 Acceptor0 SelectChannelConnector@0.0.0.0:64726, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   29) Thread[id=407, name=qtp279283093-407 Acceptor1 SelectChannelConnector@0.0.0.0:35826, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   30) Thread[id=409, name=qtp279283093-409 Acceptor3 SelectChannelConnector@0.0.0.0:35826, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   31) Thread[id=375, name=qtp1166599758-375 Acceptor3 SelectChannelConnector@0.0.0.0:16534, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   32) Thread[id=408, name=qtp279283093-408 Acceptor2 SelectChannelConnector@0.0.0.0:35826, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   33) Thread[id=436, name=qtp1951812991-436 Acceptor3 SelectChannelConnector@0.0.0.0:46423, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   34) Thread[id=374, name=qtp1166599758-374 Acceptor2 SelectChannelConnector@0.0.0.0:16534, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   35) Thread[id=387, name=qtp1396210494-387 Acceptor1 SelectChannelConnector@0.0.0.0:64726, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   36) Thread[id=435, name=qtp1951812991-435 Acceptor2 SelectChannelConnector@0.0.0.0:46423, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([E10CCE4D9DEDE79A]:0)
[junit4:junit4] Completed on J0 in 38.90s, 1 test, 2 errors <<< FAILURES!

[...truncated 116 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:335: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/module-build.xml:61: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:1163: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:827: There were test failures: 233 suites, 970 tests, 4 suite-level errors, 807 ignored

Total time: 20 minutes 47 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure

[JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 613 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.

Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/613/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
12 threads leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:     1) Thread[id=185, name=qtp507512929-185 Acceptor2 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=183, name=qtp507512929-183 Acceptor0 SelectChannelConnector@0.0.0.0:24328, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=165, name=qtp15205627-165 Acceptor1 SelectChannelConnector@0.0.0.0:18910, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=115, name=qtp1065037894-115 Acceptor0 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=184, name=qtp507512929-184 Acceptor1 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=118, name=qtp1065037894-118 Acceptor3 SelectChannelConnector@0.0.0.0:38672, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=166, name=qtp15205627-166 Acceptor2 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=117, name=qtp1065037894-117 Acceptor2 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=186, name=qtp507512929-186 Acceptor3 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=167, name=qtp15205627-167 Acceptor3 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=116, name=qtp1065037894-116 Acceptor1 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=164, name=qtp15205627-164 Acceptor0 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 12 threads leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest: 
   1) Thread[id=185, name=qtp507512929-185 Acceptor2 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=183, name=qtp507512929-183 Acceptor0 SelectChannelConnector@0.0.0.0:24328, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=165, name=qtp15205627-165 Acceptor1 SelectChannelConnector@0.0.0.0:18910, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=115, name=qtp1065037894-115 Acceptor0 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=184, name=qtp507512929-184 Acceptor1 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=118, name=qtp1065037894-118 Acceptor3 SelectChannelConnector@0.0.0.0:38672, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=166, name=qtp15205627-166 Acceptor2 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=117, name=qtp1065037894-117 Acceptor2 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=186, name=qtp507512929-186 Acceptor3 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=167, name=qtp15205627-167 Acceptor3 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=116, name=qtp1065037894-116 Acceptor1 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=164, name=qtp15205627-164 Acceptor0 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([C6F54FC7A3B9989F]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=185, name=qtp507512929-185 Acceptor2 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=183, name=qtp507512929-183 Acceptor0 SelectChannelConnector@0.0.0.0:24328, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=117, name=qtp1065037894-117 Acceptor2 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=165, name=qtp15205627-165 Acceptor1 SelectChannelConnector@0.0.0.0:18910, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=186, name=qtp507512929-186 Acceptor3 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=167, name=qtp15205627-167 Acceptor3 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=115, name=qtp1065037894-115 Acceptor0 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=184, name=qtp507512929-184 Acceptor1 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=116, name=qtp1065037894-116 Acceptor1 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=164, name=qtp15205627-164 Acceptor0 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=118, name=qtp1065037894-118 Acceptor3 SelectChannelConnector@0.0.0.0:38672, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=166, name=qtp15205627-166 Acceptor2 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=185, name=qtp507512929-185 Acceptor2 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=183, name=qtp507512929-183 Acceptor0 SelectChannelConnector@0.0.0.0:24328, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=117, name=qtp1065037894-117 Acceptor2 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=165, name=qtp15205627-165 Acceptor1 SelectChannelConnector@0.0.0.0:18910, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=186, name=qtp507512929-186 Acceptor3 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=167, name=qtp15205627-167 Acceptor3 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=115, name=qtp1065037894-115 Acceptor0 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=184, name=qtp507512929-184 Acceptor1 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=116, name=qtp1065037894-116 Acceptor1 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=164, name=qtp15205627-164 Acceptor0 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=118, name=qtp1065037894-118 Acceptor3 SelectChannelConnector@0.0.0.0:38672, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=166, name=qtp15205627-166 Acceptor2 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([C6F54FC7A3B9989F]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.request.TestRemoteStreaming

Error Message:
4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming:     1) Thread[id=329, name=qtp651274847-329 Acceptor1 SelectChannelConnector@0.0.0.0:६३०६६, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=331, name=qtp651274847-331 Acceptor3 SelectChannelConnector@0.0.0.0:६३०६६, state=RUNNABLE, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=330, name=qtp651274847-330 Acceptor2 SelectChannelConnector@0.0.0.0:६३०६६, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=328, name=qtp651274847-328 Acceptor0 SelectChannelConnector@0.0.0.0:६३०६६, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming: 
   1) Thread[id=329, name=qtp651274847-329 Acceptor1 SelectChannelConnector@0.0.0.0:६३०६६, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=331, name=qtp651274847-331 Acceptor3 SelectChannelConnector@0.0.0.0:६३०६६, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=330, name=qtp651274847-330 Acceptor2 SelectChannelConnector@0.0.0.0:६३०६६, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=328, name=qtp651274847-328 Acceptor0 SelectChannelConnector@0.0.0.0:६३०६६, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([C6F54FC7A3B9989F]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.request.TestRemoteStreaming

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=329, name=qtp651274847-329 Acceptor1 SelectChannelConnector@0.0.0.0:६३०६६, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=331, name=qtp651274847-331 Acceptor3 SelectChannelConnector@0.0.0.0:६३०६६, state=RUNNABLE, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=330, name=qtp651274847-330 Acceptor2 SelectChannelConnector@0.0.0.0:६३०६६, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=328, name=qtp651274847-328 Acceptor0 SelectChannelConnector@0.0.0.0:६३०६६, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=329, name=qtp651274847-329 Acceptor1 SelectChannelConnector@0.0.0.0:६३०६६, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=331, name=qtp651274847-331 Acceptor3 SelectChannelConnector@0.0.0.0:६३०६६, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=330, name=qtp651274847-330 Acceptor2 SelectChannelConnector@0.0.0.0:६३०६६, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=328, name=qtp651274847-328 Acceptor0 SelectChannelConnector@0.0.0.0:६३०६६, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([C6F54FC7A3B9989F]:0)




Build Log:
[...truncated 8416 lines...]
[junit4:junit4] Suite: org.apache.solr.request.TestRemoteStreaming
[junit4:junit4]   2> 0 T321 oas.SolrTestCaseJ4.initCore ####initCore
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-TestRemoteStreaming-1352218923210
[junit4:junit4]   2> 1 T321 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4:junit4]   2> 571 T321 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 951 T321 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:?????
[junit4:junit4]   2> 997 T321 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 997 T321 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: solr/
[junit4:junit4]   2> 998 T321 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: 'solr/'
[junit4:junit4]   2> 1045 T321 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1045 T321 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1046 T321 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: solr/
[junit4:junit4]   2> 1046 T321 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solr/solr.xml
[junit4:junit4]   2> 1046 T321 oasc.CoreContainer.<init> New CoreContainer 601419896
[junit4:junit4]   2> 1047 T321 oasc.CoreContainer$Initializer.initialize no solr.xml file found - using default
[junit4:junit4]   2> 1047 T321 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'solr/'
[junit4:junit4]   2> 1048 T321 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'solr/'
[junit4:junit4]   2> 1096 T321 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 1123 T321 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: solr/collection1
[junit4:junit4]   2> 1125 T321 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'solr/collection1/'
[junit4:junit4]   2> 1211 T321 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 1285 T321 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1291 T321 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 1302 T321 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1930 T321 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1942 T321 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1947 T321 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1978 T321 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1987 T321 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1992 T321 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1997 T321 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1999 T321 oasc.SolrCore.<init> [collection1] Opening new SolrCore at solr/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-TestRemoteStreaming-1352218923210/
[junit4:junit4]   2> 1999 T321 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@732d14e0
[junit4:junit4]   2> 2001 T321 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-TestRemoteStreaming-1352218923210/index/
[junit4:junit4]   2> 2001 T321 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-TestRemoteStreaming-1352218923210/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2003 T321 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-TestRemoteStreaming-1352218923210/index forceNew:false
[junit4:junit4]   2> 2005 T321 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@257938cb lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c9c5420),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 2005 T321 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 2006 T321 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 2009 T321 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2009 T321 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 2010 T321 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2010 T321 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 2011 T321 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2011 T321 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2012 T321 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2012 T321 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2013 T321 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2014 T321 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 2014 T321 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 2015 T321 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2015 T321 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 2016 T321 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 2016 T321 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 2017 T321 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2017 T321 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2018 T321 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2018 T321 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2019 T321 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2020 T321 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2020 T321 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2021 T321 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2021 T321 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 2022 T321 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 2034 T321 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2039 T321 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 2041 T321 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: []
[junit4:junit4]   2> 2044 T321 oass.SolrIndexSearcher.<init> Opening Searcher@11c9a329 main
[junit4:junit4]   2> 2044 T321 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 2045 T321 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2045 T321 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2046 T321 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 2079 T321 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 2123 T321 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 2124 T321 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 2124 T321 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 2124 T321 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 2124 T321 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 2125 T321 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 2125 T321 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 2125 T321 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 2125 T321 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 2126 T321 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 2141 T336 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@11c9a329 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2144 T321 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 2145 T321 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 2145 T321 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 2145 T321 oas.SolrJettyTestBase.createJetty Jetty Assigned Port#63066
[junit4:junit4]   2> 2151 T321 oas.SolrTestCaseJ4.setUp ###Starting testQtUpdateFails
[junit4:junit4]   2> 2209 T321 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C29 name=collection1 org.apache.solr.core.SolrCore@3e5d2347
[junit4:junit4]   2> 3090 T334 C29 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@257938cb lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c9c5420),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 3091 T334 C29 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 3098 T334 C29 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1234]} 0 17
[junit4:junit4]   2> 3228 T335 C29 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 3232 T335 C29 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@257938cb lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c9c5420),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@257938cb lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c9c5420),segFN=segments_2,generation=2,filenames=[_0_4.len, _0_1.len, _0_3.len, _0_5.len, segments_2, _0.inf, _0.si, _0.pst, _0.fld]
[junit4:junit4]   2> 3232 T335 C29 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 3237 T335 C29 oass.SolrIndexSearcher.<init> Opening Searcher@704c8f74 main
[junit4:junit4]   2> 3238 T335 C29 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3238 T335 C29 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 3240 T336 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@704c8f74 main{StandardDirectoryReader(segments_2:3 _0(4.1):C1)}
[junit4:junit4]   2> 3240 T335 C29 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 12
[junit4:junit4]   2> 3300 T333 C29 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=36 
[junit4:junit4]   2> 3308 T321 oas.SolrTestCaseJ4.tearDown ###Ending testQtUpdateFails
[junit4:junit4]   2> 3314 T321 oas.SolrTestCaseJ4.setUp ###Starting testMakeDeleteAllUrl
[junit4:junit4]   2> 3319 T335 C29 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1234]} 0 2
[junit4:junit4]   2> 3322 T333 C29 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 3331 T333 C29 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@257938cb lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c9c5420),segFN=segments_2,generation=2,filenames=[_0_4.len, _0_1.len, _0_3.len, _0_5.len, segments_2, _0.inf, _0.si, _0.pst, _0.fld]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@257938cb lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c9c5420),segFN=segments_3,generation=3,filenames=[_1_1.len, _1.pst, _1.fld, _1_3.len, _1.si, _1.inf, _1_4.len, _1_5.len, segments_3]
[junit4:junit4]   2> 3331 T333 C29 oasc.SolrDeletionPolicy.updateCommits newest commit = 3
[junit4:junit4]   2> 3333 T333 C29 oass.SolrIndexSearcher.<init> Opening Searcher@39a24033 main
[junit4:junit4]   2> 3333 T333 C29 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3333 T333 C29 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 3335 T336 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@39a24033 main{StandardDirectoryReader(segments_3:7 _1(4.1):C1)}
[junit4:junit4]   2> 3335 T333 C29 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 14
[junit4:junit4]   2> 3339 T334 C29 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 3347 T333 C29 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4:junit4]   2> 3348 T333 C29 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 3348 T333 C29 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@257938cb lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c9c5420),segFN=segments_3,generation=3,filenames=[_1_1.len, _1.pst, _1.fld, _1_3.len, _1.si, _1.inf, _1_4.len, _1_5.len, segments_3]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@257938cb lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c9c5420),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 3348 T333 C29 oasc.SolrDeletionPolicy.updateCommits newest commit = 4
[junit4:junit4]   2> 3349 T333 C29 oass.SolrIndexSearcher.<init> Opening Searcher@541eae08 main
[junit4:junit4]   2> 3349 T333 C29 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3350 T333 C29 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 3351 T336 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@541eae08 main{StandardDirectoryReader(segments_4:8)}
[junit4:junit4]   2> 3351 T333 C29 UPDATE [collection1] webapp=/solr path=/update params={commit=true&stream.body=<delete><query>*:*</query></delete>} {deleteByQuery=*:*,commit=} 0 4
[junit4:junit4]   2> 3365 T334 C29 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 3366 T321 oas.SolrTestCaseJ4.tearDown ###Ending testMakeDeleteAllUrl
[junit4:junit4]   2> 3372 T321 oas.SolrTestCaseJ4.setUp ###Starting testNoUrlAccess
[junit4:junit4]   2> 3377 T333 C29 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1234]} 0 1
[junit4:junit4]   2> 3381 T335 C29 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 3383 T335 C29 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@257938cb lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c9c5420),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@257938cb lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c9c5420),segFN=segments_5,generation=5,filenames=[_2.si, _2.pst, _2.fld, _2_5.len, _2.inf, _2_3.len, segments_5, _2_1.len, _2_4.len]
[junit4:junit4]   2> 3384 T335 C29 oasc.SolrDeletionPolicy.updateCommits newest commit = 5
[junit4:junit4]   2> 3436 T335 C29 oass.SolrIndexSearcher.<init> Opening Searcher@e71d24c main
[junit4:junit4]   2> 3436 T335 C29 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3437 T335 C29 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 3439 T336 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@e71d24c main{StandardDirectoryReader(segments_5:10 _2(4.1):C1)}
[junit4:junit4]   2> 3439 T335 C29 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 58
[junit4:junit4]   2> 3443 T333 C29 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 3448 T335 C29 REQ [collection1] webapp=/solr path=/select params={wt=javabin&stream.url=http://127.0.0.1:63066/solr/update?commit%3Dtrue%26stream.body%3D%253Cdelete%253E%253Cquery%253E*%253A*%253C%252Fquery%253E%253C%252Fdelete%253E&q=*:*&version=2} hits=1 status=0 QTime=0 
[junit4:junit4]   2> 3451 T334 C29 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=0 
[junit4:junit4]   2> 3452 T321 oas.SolrTestCaseJ4.tearDown ###Ending testNoUrlAccess
[junit4:junit4]   2> 3458 T321 oas.SolrTestCaseJ4.setUp ###Starting testStreamUrl
[junit4:junit4]   2> 3461 T333 C29 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1234]} 0 1
[junit4:junit4]   2> 3465 T335 C29 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 3519 T335 C29 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@257938cb lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c9c5420),segFN=segments_5,generation=5,filenames=[_2.si, _2.pst, _2.fld, _2_5.len, _2.inf, _2_3.len, segments_5, _2_1.len, _2_4.len]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@257938cb lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c9c5420),segFN=segments_6,generation=6,filenames=[_3.fld, _3.inf, _3_5.len, _3_4.len, _3.si, _3_3.len, segments_6, _3.pst, _3_1.len]
[junit4:junit4]   2> 3519 T335 C29 oasc.SolrDeletionPolicy.updateCommits newest commit = 6
[junit4:junit4]   2> 3521 T335 C29 oass.SolrIndexSearcher.<init> Opening Searcher@3f28ad90 main
[junit4:junit4]   2> 3521 T335 C29 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3521 T335 C29 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 3523 T336 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3f28ad90 main{StandardDirectoryReader(segments_6:14 _3(4.1):C1)}
[junit4:junit4]   2> 3524 T335 C29 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 59
[junit4:junit4]   2> 3527 T334 C29 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 3532 T335 C29 REQ [collection1] webapp=/solr path=/select params={wt=csv&q=*:*&fl=id} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 3553 T333 C29 REQ [collection1] webapp=/solr path=/debug/dump params={wt=xml&stream.url=http://127.0.0.1:63066/solr/select?q%3D*:*%26fl%3Did%26wt%3Dcsv} status=0 QTime=24 
[junit4:junit4]   2> 3554 T321 oas.SolrTestCaseJ4.tearDown ###Ending testStreamUrl
[junit4:junit4]   2> 3601 T321 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=601419896
[junit4:junit4]   2> 3601 T321 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@3e5d2347
[junit4:junit4]   2> 3611 T321 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=5,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=4,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4]   2> 3611 T321 oasc.SolrCore.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 3612 T321 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 3612 T321 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 3613 T321 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 3634 T321 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 3736 T321 oejut.QueuedThreadPool.doStop WARNING 4 threads could not be stopped
[junit4:junit4]   2> 3736 T321 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> 3772 T320 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 4 leaked thread(s).
[junit4:junit4]   2> 23927 T320 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming: 
[junit4:junit4]   2> 	   1) Thread[id=329, name=qtp651274847-329 Acceptor1 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   2) Thread[id=331, name=qtp651274847-331 Acceptor3 SelectChannelConnector@0.0.0.0:?????, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   3) Thread[id=330, name=qtp651274847-330 Acceptor2 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   4) Thread[id=328, name=qtp651274847-328 Acceptor0 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 23928 T320 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
[junit4:junit4]   2> 	   1) Thread[id=329, name=qtp651274847-329 Acceptor1 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	   2) Thread[id=331, name=qtp651274847-331 Acceptor3 SelectChannelConnector@0.0.0.0:?????, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	   3) Thread[id=330, name=qtp651274847-330 Acceptor2 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	   4) Thread[id=328, name=qtp651274847-328 Acceptor0 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 26971 T320 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
[junit4:junit4]   2> 	   1) Thread[id=329, name=qtp651274847-329 Acceptor1 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   2) Thread[id=331, name=qtp651274847-331 Acceptor3 SelectChannelConnector@0.0.0.0:?????, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   3) Thread[id=330, name=qtp651274847-330 Acceptor2 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   4) Thread[id=328, name=qtp651274847-328 Acceptor0 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=SimpleText, sim=DefaultSimilarity, locale=hi_IN, timezone=Canada/Yukon
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=5,free=204122560,total=394264576
[junit4:junit4]   2> NOTE: All tests run in this JVM: [SolrInfoMBeanTest, StandardRequestHandlerTest, TestUtils, TestCharFilters, TestBM25SimilarityFactory, TestSurroundQueryParser, TestStressVersions, DocumentAnalysisRequestHandlerTest, DebugComponentTest, TestRealTimeGet, TestFastOutputStream, PrimitiveFieldTypeTest, TestCodecSupport, TestUpdate, TestPropInject, TestRecovery, TestJmxIntegration, XmlUpdateRequestHandlerTest, TestFunctionQuery, ZkCLITest, TermVectorComponentTest, MultiTermTest, EchoParamsTest, SolrCoreTest, TestRemoteStreaming]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestRemoteStreaming -Dtests.seed=C6F54FC7A3B9989F -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=hi_IN -Dtests.timezone=Canada/Yukon -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR   0.00s J1 | TestRemoteStreaming (suite) <<<
[junit4:junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming: 
[junit4:junit4]    >    1) Thread[id=329, name=qtp651274847-329 Acceptor1 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=331, name=qtp651274847-331 Acceptor3 SelectChannelConnector@0.0.0.0:?????, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=330, name=qtp651274847-330 Acceptor2 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=328, name=qtp651274847-328 Acceptor0 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([C6F54FC7A3B9989F]:0)
[junit4:junit4]    > Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4]    >    1) Thread[id=329, name=qtp651274847-329 Acceptor1 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=331, name=qtp651274847-331 Acceptor3 SelectChannelConnector@0.0.0.0:?????, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=330, name=qtp651274847-330 Acceptor2 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=328, name=qtp651274847-328 Acceptor0 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([C6F54FC7A3B9989F]:0)
[junit4:junit4] Completed on J1 in 27.19s, 4 tests, 2 errors <<< FAILURES!

[...truncated 452 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4]   2> 4 T99 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1352218906704
[junit4:junit4]   2> 10 T99 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 103 T100 oaz.Environment.logEnv Server environment:zookeeper.version=3.3.6-1366786, built on 07/29/2012 06:22 GMT
[junit4:junit4]   2> 104 T100 oaz.Environment.logEnv Server environment:host.name=lucene.zones.apache.org
[junit4:junit4]   2> 104 T100 oaz.Environment.logEnv Server environment:java.version=1.7.0_09
[junit4:junit4]   2> 104 T100 oaz.Environment.logEnv Server environment:java.vendor=Oracle Corporation
[junit4:junit4]   2> 104 T100 oaz.Environment.logEnv Server environment:java.home=/usr/local/openjdk7/jre
[junit4:junit4]   2> 104 T100 oaz.Environment.logEnv Server environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/common/lucene-analyzers-common-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/highlighter/lucene-highlighter-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/memory/lucene-memory-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/misc/lucene-misc-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/spatial/lucene-spatial-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/suggest/lucene-suggest-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/grouping/lucene-grouping-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queries/lucene-queries-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queryparser/lucene-queryparser-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/easymock-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/guava-r05.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/log4j-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/zookeeper-3.3.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-continuation-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-deploy-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-http-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-io-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-jmx-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-security-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-server-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-servlet-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-util-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-webapp-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-xml-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-javamail.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jdepend.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.4.jar
[junit4:junit4]   2> 105 T100 oaz.Environment.logEnv Server environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
[junit4:junit4]   2> 105 T100 oaz.Environment.logEnv Server environment:java.io.tmpdir=.
[junit4:junit4]   2> 106 T100 oaz.Environment.logEnv Server environment:java.compiler=<NA>
[junit4:junit4]   2> 106 T100 oaz.Environment.logEnv Server environment:os.name=FreeBSD
[junit4:junit4]   2> 106 T100 oaz.Environment.logEnv Server environment:os.arch=amd64
[junit4:junit4]   2> 106 T100 oaz.Environment.logEnv Server environment:os.version=9.0-RELEASE
[junit4:junit4]   2> 106 T100 oaz.Environment.logEnv Server environment:user.name=hudson
[junit4:junit4]   2> 106 T100 oaz.Environment.logEnv Server environment:user.home=/home/hudson
[junit4:junit4]   2> 107 T100 oaz.Environment.logEnv Server environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 143 T100 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 173 T100 oazs.NIOServerCnxn$Factory.<init> binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 218 T99 oasc.ZkTestServer.run start zk server on port:64775
[junit4:junit4]   2> 236 T100 oazsp.FileTxnSnapLog.save Snapshotting: 0
[junit4:junit4]   2> 308 T99 oaz.Environment.logEnv Client environment:zookeeper.version=3.3.6-1366786, built on 07/29/2012 06:22 GMT
[junit4:junit4]   2> 308 T99 oaz.Environment.logEnv Client environment:host.name=lucene.zones.apache.org
[junit4:junit4]   2> 309 T99 oaz.Environment.logEnv Client environment:java.version=1.7.0_09
[junit4:junit4]   2> 309 T99 oaz.Environment.logEnv Client environment:java.vendor=Oracle Corporation
[junit4:junit4]   2> 309 T99 oaz.Environment.logEnv Client environment:java.home=/usr/local/openjdk7/jre
[junit4:junit4]   2> 310 T99 oaz.Environment.logEnv Client environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/common/lucene-analyzers-common-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/highlighter/lucene-highlighter-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/memory/lucene-memory-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/misc/lucene-misc-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/spatial/lucene-spatial-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/suggest/lucene-suggest-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/grouping/lucene-grouping-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queries/lucene-queries-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queryparser/lucene-queryparser-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/easymock-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/guava-r05.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/log4j-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/zookeeper-3.3.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-continuation-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-deploy-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-http-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-io-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-jmx-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-security-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-server-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-servlet-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-util-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-webapp-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-xml-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-javamail.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jdepend.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.4.jar
[junit4:junit4]   2> 311 T99 oaz.Environment.logEnv Client environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
[junit4:junit4]   2> 311 T99 oaz.Environment.logEnv Client environment:java.io.tmpdir=.
[junit4:junit4]   2> 311 T99 oaz.Environment.logEnv Client environment:java.compiler=<NA>
[junit4:junit4]   2> 312 T99 oaz.Environment.logEnv Client environment:os.name=FreeBSD
[junit4:junit4]   2> 312 T99 oaz.Environment.logEnv Client environment:os.arch=amd64
[junit4:junit4]   2> 312 T99 oaz.Environment.logEnv Client environment:os.version=9.0-RELEASE
[junit4:junit4]   2> 313 T99 oaz.Environment.logEnv Client environment:user.name=hudson
[junit4:junit4]   2> 313 T99 oaz.Environment.logEnv Client environment:user.home=/home/hudson
[junit4:junit4]   2> 313 T99 oaz.Environment.logEnv Client environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 319 T99 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:64775 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@5d1ae1c3
[junit4:junit4]   2> 374 T104 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:64775
[junit4:junit4]   2> 375 T99 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 378 T104 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:64775, initiating session
[junit4:junit4]   2> 378 T101 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:51518
[junit4:junit4]   2> 394 T101 oazs.NIOServerCnxn.doIO WARNING Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
[junit4:junit4]   2> 394 T101 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:51518 (no session established for client)
[junit4:junit4]   2> 395 T104 oaz.ClientCnxn$SendThread.run Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
[junit4:junit4]   2> 2127 T104 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost/127.0.0.1:64775
[junit4:junit4]   2> 2128 T104 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:64775, initiating session
[junit4:junit4]   2> 2128 T101 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:21267
[junit4:junit4]   2> 2129 T101 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:21267
[junit4:junit4]   2> 2144 T103 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 2176 T103 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad687116a0000 with negotiated timeout 10000 for client /140.211.11.196:21267
[junit4:junit4]   2> 2176 T104 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:64775, sessionid = 0x13ad687116a0000, negotiated timeout = 10000
[junit4:junit4]   2> 2180 T106 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5d1ae1c3 name:ZooKeeperConnection Watcher:127.0.0.1:64775 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2180 T99 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2185 T99 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 2252 T105 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x13ad687116a0000
[junit4:junit4]   2> 2253 T106 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 2253 T99 oaz.ZooKeeper.close Session: 0x13ad687116a0000 closed
[junit4:junit4]   2> 2254 T99 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:64775/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@13c77dd0
[junit4:junit4]   2> 2254 T101 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:21267 which had sessionid 0x13ad687116a0000
[junit4:junit4]   2> 2255 T107 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:64775
[junit4:junit4]   2> 2255 T99 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2256 T107 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:64775, initiating session
[junit4:junit4]   2> 2256 T101 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:35059
[junit4:junit4]   2> 2256 T101 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:35059
[junit4:junit4]   2> 2258 T103 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad687116a0001 with negotiated timeout 10000 for client /140.211.11.196:35059
[junit4:junit4]   2> 2258 T107 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:64775, sessionid = 0x13ad687116a0001, negotiated timeout = 10000
[junit4:junit4]   2> 2258 T108 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13c77dd0 name:ZooKeeperConnection Watcher:127.0.0.1:64775/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2259 T99 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2268 T99 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 2273 T99 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 2283 T99 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 2286 T99 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 2289 T99 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2318 T99 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2336 T99 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 2337 T99 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 2442 T99 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2443 T99 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2481 T99 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 2481 T99 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 2486 T99 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 2486 T99 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 2489 T99 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 2490 T99 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 2494 T99 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 2494 T99 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 2497 T99 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 2499 T99 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 2502 T99 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 2502 T99 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 2505 T99 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 2506 T99 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 2509 T105 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x13ad687116a0001
[junit4:junit4]   2> 2517 T108 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 2517 T101 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:35059 which had sessionid 0x13ad687116a0001
[junit4:junit4]   2> 2517 T99 oaz.ZooKeeper.close Session: 0x13ad687116a0001 closed
[junit4:junit4]   2> 2980 T99 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 3245 T99 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:38672
[junit4:junit4]   2> 3277 T99 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3278 T99 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 3278 T99 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 3315 T99 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 3316 T99 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3316 T99 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 3316 T99 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 3317 T99 oasc.CoreContainer.<init> New CoreContainer 1200082580
[junit4:junit4]   2> 3317 T99 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 3318 T99 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 3363 T99 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 3384 T99 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:64775/solr
[junit4:junit4]   2> 3420 T99 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:64775/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@198cc898
[junit4:junit4]   2> 3421 T119 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:64775
[junit4:junit4]   2> 3421 T99 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3422 T119 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:64775, initiating session
[junit4:junit4]   2> 3422 T101 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:17343
[junit4:junit4]   2> 3423 T101 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:17343
[junit4:junit4]   2> 3424 T103 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad687116a0002 with negotiated timeout 5000 for client /140.211.11.196:17343
[junit4:junit4]   2> 3424 T119 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:64775, sessionid = 0x13ad687116a0002, negotiated timeout = 5000
[junit4:junit4]   2> 3425 T120 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@198cc898 name:ZooKeeperConnection Watcher:127.0.0.1:64775/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3425 T99 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3445 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0002 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3456 T99 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 3459 T99 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38672_solr
[junit4:junit4]   2> 3465 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0002 type:delete cxid:0x7 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:38672_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:38672_solr
[junit4:junit4]   2> 3466 T99 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38672_solr
[junit4:junit4]   2> 3469 T99 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 3470 T99 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 3470 T99 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 3470 T99 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 3471 T99 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 3471 T99 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 3471 T99 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 3472 T99 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 3472 T99 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 3473 T99 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 3487 T99 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 3506 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0002 type:delete cxid:0x15 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 3507 T99 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 3510 T99 oasc.Overseer.start Overseer (id=88619018288234498-127.0.0.1:38672_solr-n_0000000000) starting
[junit4:junit4]   2> 3510 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0002 type:create cxid:0x1a zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3526 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0002 type:create cxid:0x1b zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3527 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0002 type:create cxid:0x1c zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3533 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0002 type:create cxid:0x1d zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3534 T122 oasc.OverseerCollectionProcessor.run Process current queue of collection messages
[junit4:junit4]   2> 3535 T99 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 3542 T99 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 3562 T121 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 3606 T99 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 3606 T99 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 3607 T99 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 3608 T99 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 3610 T99 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 3611 T99 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 3611 T99 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 3693 T99 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 3743 T99 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 3844 T99 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 3852 T99 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 4296 T99 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 4306 T99 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 4309 T99 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 4329 T99 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4334 T99 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4339 T99 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4342 T99 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4345 T99 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/control/data/
[junit4:junit4]   2> 4345 T99 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2691cde4
[junit4:junit4]   2> 4370 T99 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/control/data/index/
[junit4:junit4]   2> 4371 T99 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 4372 T99 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/control/data/index forceNew:false
[junit4:junit4]   2> 4382 T99 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/control/data/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 4382 T99 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 4383 T99 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 4385 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 4386 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 4386 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 4387 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 4387 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 4388 T99 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 4388 T99 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 4389 T99 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 4390 T99 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 4390 T99 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 4391 T99 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 4392 T99 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 4392 T99 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 4393 T99 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 4393 T99 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 4394 T99 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 4394 T99 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4395 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4396 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4396 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4397 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4397 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4398 T99 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4399 T99 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 4399 T99 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 4408 T99 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 4412 T99 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 4413 T99 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 4417 T99 oass.SolrIndexSearcher.<init> Opening Searcher@603678ef main
[junit4:junit4]   2> 4418 T99 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 4418 T99 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 4419 T99 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 4433 T99 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 4503 T126 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@603678ef main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 4508 T99 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 4509 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0002 type:create cxid:0x53 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 4522 T99 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 4523 T99 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:38672/solr shardId:control_shard
[junit4:junit4]   2> 4532 T99 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 4546 T99 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/control_shard/election
[junit4:junit4]   2> 4573 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0002 type:delete cxid:0x64 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 4575 T99 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 4575 T121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4576 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0002 type:create cxid:0x67 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4588 T121 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"control_shard",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:38672_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:38672/solr"}
[junit4:junit4]   2> 4600 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0002 type:create cxid:0x6a zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 4644 T120 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (1)
[junit4:junit4]   2> 5097 T99 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 5097 T99 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 5098 T99 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:38672/solr/collection1/
[junit4:junit4]   2> 5099 T99 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 5099 T99 oasc.SyncStrategy.syncToMe http://127.0.0.1:38672/solr/collection1/ has no replicas
[junit4:junit4]   2> 5100 T99 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:38672/solr/collection1/
[junit4:junit4]   2> 5101 T99 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/control_shard
[junit4:junit4]   2> 5112 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0002 type:create cxid:0x88 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5157 T121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5170 T120 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (1)
[junit4:junit4]   2> 5229 T99 oasc.ZkController.register We are http://127.0.0.1:38672/solr/collection1/ and leader is http://127.0.0.1:38672/solr/collection1/
[junit4:junit4]   2> 5229 T99 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:38672/solr
[junit4:junit4]   2> 5230 T99 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 5230 T99 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 5233 T99 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5234 T99 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 5235 T99 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 5292 T99 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 5440 T99 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 5443 T99 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:19972
[junit4:junit4]   2> 5443 T99 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 5444 T99 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352218912010
[junit4:junit4]   2> 5444 T99 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352218912010/'
[junit4:junit4]   2> 5482 T99 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 5482 T99 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 5483 T99 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352218912010
[junit4:junit4]   2> 5483 T99 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352218912010/solr.xml
[junit4:junit4]   2> 5483 T99 oasc.CoreContainer.<init> New CoreContainer 1719203423
[junit4:junit4]   2> 5484 T99 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352218912010/'
[junit4:junit4]   2> 5484 T99 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352218912010/'
[junit4:junit4]   2> 5508 T99 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 5519 T99 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:64775/solr
[junit4:junit4]   2> 5520 T99 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:64775/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@45d5584a
[junit4:junit4]   2> 5521 T136 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:64775
[junit4:junit4]   2> 5521 T99 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 5522 T136 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:64775, initiating session
[junit4:junit4]   2> 5522 T101 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:24932
[junit4:junit4]   2> 5523 T101 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:24932
[junit4:junit4]   2> 5533 T103 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad687116a0003 with negotiated timeout 5000 for client /140.211.11.196:24932
[junit4:junit4]   2> 5533 T136 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:64775, sessionid = 0x13ad687116a0003, negotiated timeout = 5000
[junit4:junit4]   2> 5534 T137 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@45d5584a name:ZooKeeperConnection Watcher:127.0.0.1:64775/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 5534 T99 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 5535 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0003 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5537 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0003 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5546 T99 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:19972_solr
[junit4:junit4]   2> 5547 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0003 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:19972_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:19972_solr
[junit4:junit4]   2> 5548 T99 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:19972_solr
[junit4:junit4]   2> 5558 T120 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 5559 T99 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 5559 T99 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 5559 T99 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 5560 T99 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 5560 T99 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 5561 T99 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 5561 T99 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 5561 T99 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 5562 T99 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 5562 T99 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 5583 T99 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 5589 T99 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352218912010/collection1
[junit4:junit4]   2> 5589 T99 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 5590 T99 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 5591 T99 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 5592 T99 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352218912010/collection1/'
[junit4:junit4]   2> 5593 T99 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty1-1352218912010/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 5594 T99 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty1-1352218912010/collection1/lib/README' to classloader
[junit4:junit4]   2> 5637 T99 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 5681 T121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5683 T121 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"control_shard",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:38672_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:38672/solr"}
[junit4:junit4]   2> 5688 T99 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 5790 T120 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 5815 T137 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 5815 T99 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 5823 T99 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 6261 T99 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 6270 T99 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 6273 T99 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 6289 T99 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6302 T99 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6307 T99 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6309 T99 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6313 T99 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352218912010/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty1/
[junit4:junit4]   2> 6313 T99 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2691cde4
[junit4:junit4]   2> 6314 T99 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty1/index/
[junit4:junit4]   2> 6315 T99 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 6316 T99 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty1/index forceNew:false
[junit4:junit4]   2> 6329 T99 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty1/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 6330 T99 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 6331 T99 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 6333 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 6333 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 6334 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 6334 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 6335 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 6335 T99 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 6336 T99 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 6337 T99 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 6337 T99 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 6338 T99 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 6339 T99 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 6339 T99 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 6340 T99 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 6340 T99 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 6341 T99 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 6341 T99 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 6342 T99 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6343 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6343 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6344 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6345 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6345 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6346 T99 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6347 T99 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 6347 T99 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 6356 T99 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 6360 T99 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 6361 T99 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 6364 T99 oass.SolrIndexSearcher.<init> Opening Searcher@7b0e6f6e main
[junit4:junit4]   2> 6365 T99 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 6365 T99 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 6366 T99 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 6381 T99 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 6434 T142 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7b0e6f6e main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 6439 T99 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6861 T121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6862 T121 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:19972_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:19972/solr"}
[junit4:junit4]   2> 6862 T121 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 6885 T120 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 6885 T137 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 6942 T99 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 6942 T99 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:19972/solr shardId:shard1
[junit4:junit4]   2> 6944 T99 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 6964 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0003 type:delete cxid:0x43 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 6965 T99 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 6966 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0003 type:create cxid:0x44 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6977 T99 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 6977 T99 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 6978 T99 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:19972/solr/collection1/
[junit4:junit4]   2> 6978 T99 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 6979 T99 oasc.SyncStrategy.syncToMe http://127.0.0.1:19972/solr/collection1/ has no replicas
[junit4:junit4]   2> 6980 T99 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:19972/solr/collection1/
[junit4:junit4]   2> 6980 T99 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 6986 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0003 type:create cxid:0x4e zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7391 T121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 7432 T120 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 7432 T137 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 7454 T99 oasc.ZkController.register We are http://127.0.0.1:19972/solr/collection1/ and leader is http://127.0.0.1:19972/solr/collection1/
[junit4:junit4]   2> 7454 T99 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:19972/solr
[junit4:junit4]   2> 7455 T99 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 7455 T99 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 7462 T99 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 7463 T99 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 7464 T99 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 7464 T99 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 7590 T99 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 7593 T99 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:64816
[junit4:junit4]   2> 7594 T99 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 7594 T99 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352218914175
[junit4:junit4]   2> 7594 T99 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352218914175/'
[junit4:junit4]   2> 7624 T99 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 7625 T99 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 7625 T99 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352218914175
[junit4:junit4]   2> 7625 T99 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352218914175/solr.xml
[junit4:junit4]   2> 7626 T99 oasc.CoreContainer.<init> New CoreContainer 764890953
[junit4:junit4]   2> 7626 T99 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352218914175/'
[junit4:junit4]   2> 7627 T99 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352218914175/'
[junit4:junit4]   2> 7651 T99 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 7662 T99 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:64775/solr
[junit4:junit4]   2> 7663 T99 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:64775/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@579af7c4
[junit4:junit4]   2> 7664 T152 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:64775
[junit4:junit4]   2> 7664 T99 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7665 T152 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:64775, initiating session
[junit4:junit4]   2> 7666 T101 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:36848
[junit4:junit4]   2> 7666 T101 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:36848
[junit4:junit4]   2> 7668 T103 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad687116a0004 with negotiated timeout 5000 for client /140.211.11.196:36848
[junit4:junit4]   2> 7668 T152 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:64775, sessionid = 0x13ad687116a0004, negotiated timeout = 5000
[junit4:junit4]   2> 7668 T153 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@579af7c4 name:ZooKeeperConnection Watcher:127.0.0.1:64775/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7669 T99 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7670 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0004 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7679 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0004 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7687 T99 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64816_solr
[junit4:junit4]   2> 7688 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0004 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:64816_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:64816_solr
[junit4:junit4]   2> 7689 T99 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:64816_solr
[junit4:junit4]   2> 7693 T137 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7693 T120 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7693 T99 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 7694 T99 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 7694 T99 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 7695 T99 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 7695 T99 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 7695 T99 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 7696 T99 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 7696 T99 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 7696 T99 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 7697 T99 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 7714 T99 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 7720 T99 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352218914175/collection1
[junit4:junit4]   2> 7720 T99 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 7721 T99 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 7721 T99 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 7723 T99 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352218914175/collection1/'
[junit4:junit4]   2> 7724 T99 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty2-1352218914175/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 7724 T99 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty2-1352218914175/collection1/lib/README' to classloader
[junit4:junit4]   2> 7763 T99 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 7831 T99 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 7932 T99 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 7939 T99 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 7946 T121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 7947 T121 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:19972_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:19972/solr"}
[junit4:junit4]   2> 7961 T137 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 7961 T120 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 7961 T153 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 8386 T99 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 8395 T99 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 8398 T99 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 8414 T99 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8419 T99 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8423 T99 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8425 T99 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8427 T99 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352218914175/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty2/
[junit4:junit4]   2> 8427 T99 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2691cde4
[junit4:junit4]   2> 8428 T99 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty2/index/
[junit4:junit4]   2> 8429 T99 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 8430 T99 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty2/index forceNew:false
[junit4:junit4]   2> 8436 T99 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty2/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 8436 T99 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 8437 T99 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 8440 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 8441 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 8442 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 8442 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 8443 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 8444 T99 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 8445 T99 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 8446 T99 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 8447 T99 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 8448 T99 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 8449 T99 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 8450 T99 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 8451 T99 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 8451 T99 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 8452 T99 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 8453 T99 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 8454 T99 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8455 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8456 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8457 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8458 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8458 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8459 T99 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8460 T99 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 8460 T99 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 8471 T99 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 8475 T99 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 8477 T99 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 8480 T99 oass.SolrIndexSearcher.<init> Opening Searcher@604f05f8 main
[junit4:junit4]   2> 8482 T99 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 8482 T99 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 8483 T99 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 8505 T99 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 8559 T158 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@604f05f8 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 8567 T99 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8969 T121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8971 T121 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:64816_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:64816/solr"}
[junit4:junit4]   2> 8985 T120 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 8985 T153 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 8985 T137 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 9071 T99 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 9071 T99 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:64816/solr shardId:shard2
[junit4:junit4]   2> 9073 T99 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 9099 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0004 type:delete cxid:0x42 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 9100 T99 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 9101 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0004 type:create cxid:0x43 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9111 T99 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 9112 T99 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 9112 T99 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:64816/solr/collection1/
[junit4:junit4]   2> 9113 T99 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 9113 T99 oasc.SyncStrategy.syncToMe http://127.0.0.1:64816/solr/collection1/ has no replicas
[junit4:junit4]   2> 9114 T99 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:64816/solr/collection1/
[junit4:junit4]   2> 9114 T99 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 9118 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0004 type:create cxid:0x4c zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9491 T121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9519 T120 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 9519 T153 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 9519 T137 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 9537 T99 oasc.ZkController.register We are http://127.0.0.1:64816/solr/collection1/ and leader is http://127.0.0.1:64816/solr/collection1/
[junit4:junit4]   2> 9537 T99 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:64816/solr
[junit4:junit4]   2> 9538 T99 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 9538 T99 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9540 T99 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9541 T99 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 9541 T99 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 9542 T99 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9673 T99 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 9676 T99 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:18910
[junit4:junit4]   2> 9677 T99 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 9677 T99 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352218916255
[junit4:junit4]   2> 9678 T99 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352218916255/'
[junit4:junit4]   2> 9719 T99 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 9719 T99 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 9720 T99 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352218916255
[junit4:junit4]   2> 9720 T99 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352218916255/solr.xml
[junit4:junit4]   2> 9720 T99 oasc.CoreContainer.<init> New CoreContainer 1780142497
[junit4:junit4]   2> 9721 T99 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352218916255/'
[junit4:junit4]   2> 9721 T99 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352218916255/'
[junit4:junit4]   2> 9749 T99 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 9762 T99 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:64775/solr
[junit4:junit4]   2> 9763 T99 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:64775/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@699ae127
[junit4:junit4]   2> 9764 T168 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:64775
[junit4:junit4]   2> 9763 T99 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 9765 T168 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:64775, initiating session
[junit4:junit4]   2> 9765 T101 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:18369
[junit4:junit4]   2> 9765 T101 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:18369
[junit4:junit4]   2> 9769 T103 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad687116a0005 with negotiated timeout 5000 for client /140.211.11.196:18369
[junit4:junit4]   2> 9769 T168 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:64775, sessionid = 0x13ad687116a0005, negotiated timeout = 5000
[junit4:junit4]   2> 9770 T169 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@699ae127 name:ZooKeeperConnection Watcher:127.0.0.1:64775/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 9770 T99 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 9771 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0005 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9772 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0005 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9774 T99 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:18910_solr
[junit4:junit4]   2> 9774 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0005 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:18910_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:18910_solr
[junit4:junit4]   2> 9783 T99 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:18910_solr
[junit4:junit4]   2> 9786 T137 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 9786 T120 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 9786 T153 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 9787 T99 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 9787 T99 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 9787 T99 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 9788 T99 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 9788 T99 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 9788 T99 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 9789 T99 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 9789 T99 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 9789 T99 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 9790 T99 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 9806 T99 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 9812 T99 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352218916255/collection1
[junit4:junit4]   2> 9813 T99 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 9813 T99 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 9814 T99 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 9815 T99 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352218916255/collection1/'
[junit4:junit4]   2> 9816 T99 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty3-1352218916255/collection1/lib/README' to classloader
[junit4:junit4]   2> 9817 T99 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty3-1352218916255/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 9862 T99 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 9917 T99 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 10018 T99 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 10026 T121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 10027 T121 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:64816_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:64816/solr"}
[junit4:junit4]   2> 10029 T99 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 10086 T137 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 10086 T120 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 10086 T169 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 10086 T153 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 10573 T99 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 10583 T99 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 10586 T99 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 10610 T99 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10618 T99 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10624 T99 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10626 T99 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10629 T99 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352218916255/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty3/
[junit4:junit4]   2> 10629 T99 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2691cde4
[junit4:junit4]   2> 10631 T99 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty3/index/
[junit4:junit4]   2> 10632 T99 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 10634 T99 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty3/index forceNew:false
[junit4:junit4]   2> 10641 T99 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty3/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 10642 T99 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 10643 T99 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 10646 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 10647 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 10647 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 10648 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 10649 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 10650 T99 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 10651 T99 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 10652 T99 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 10653 T99 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 10654 T99 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 10655 T99 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr

[...truncated too long message...]

r.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=117, name=qtp1065037894-117 Acceptor2 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=165, name=qtp15205627-165 Acceptor1 SelectChannelConnector@0.0.0.0:18910, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    5) Thread[id=186, name=qtp507512929-186 Acceptor3 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    6) Thread[id=167, name=qtp15205627-167 Acceptor3 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    7) Thread[id=115, name=qtp1065037894-115 Acceptor0 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    8) Thread[id=184, name=qtp507512929-184 Acceptor1 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    9) Thread[id=116, name=qtp1065037894-116 Acceptor1 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   10) Thread[id=164, name=qtp15205627-164 Acceptor0 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   11) Thread[id=118, name=qtp1065037894-118 Acceptor3 SelectChannelConnector@0.0.0.0:38672, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   12) Thread[id=166, name=qtp15205627-166 Acceptor2 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([C6F54FC7A3B9989F]:0)
[junit4:junit4] Completed on J0 in 292.06s, 1 test, 2 errors <<< FAILURES!

[...truncated 149 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:335: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/module-build.xml:61: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:1163: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:827: There were test failures: 233 suites, 965 tests, 4 suite-level errors, 744 ignored (1 assumption)

Total time: 22 minutes 59 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure

[JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 612 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.

Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/612/

5 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZk2Test

Error Message:
16 threads leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZk2Test:     1) Thread[id=96, name=qtp1946791897-96 Acceptor1 SelectChannelConnector@0.0.0.0:32207, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=47, name=qtp526921047-47 Acceptor2 SelectChannelConnector@0.0.0.0:54171, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=30, name=qtp399389739-30 Acceptor3 SelectChannelConnector@0.0.0.0:23671, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=97, name=qtp1946791897-97 Acceptor2 SelectChannelConnector@0.0.0.0:32207, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=95, name=qtp1946791897-95 Acceptor0 SelectChannelConnector@0.0.0.0:32207, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=79, name=qtp1070454762-79 Acceptor2 SelectChannelConnector@0.0.0.0:37521, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=77, name=qtp1070454762-77 Acceptor0 SelectChannelConnector@0.0.0.0:37521, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=29, name=qtp399389739-29 Acceptor2 SelectChannelConnector@0.0.0.0:23671, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=98, name=qtp1946791897-98 Acceptor3 SelectChannelConnector@0.0.0.0:32207, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=80, name=qtp1070454762-80 Acceptor3 SelectChannelConnector@0.0.0.0:37521, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=78, name=qtp1070454762-78 Acceptor1 SelectChannelConnector@0.0.0.0:37521, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=45, name=qtp526921047-45 Acceptor0 SelectChannelConnector@0.0.0.0:54171, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=28, name=qtp399389739-28 Acceptor1 SelectChannelConnector@0.0.0.0:23671, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=48, name=qtp526921047-48 Acceptor3 SelectChannelConnector@0.0.0.0:54171, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=46, name=qtp526921047-46 Acceptor1 SelectChannelConnector@0.0.0.0:54171, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=27, name=qtp399389739-27 Acceptor0 SelectChannelConnector@0.0.0.0:23671, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 16 threads leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZk2Test: 
   1) Thread[id=96, name=qtp1946791897-96 Acceptor1 SelectChannelConnector@0.0.0.0:32207, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=47, name=qtp526921047-47 Acceptor2 SelectChannelConnector@0.0.0.0:54171, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=30, name=qtp399389739-30 Acceptor3 SelectChannelConnector@0.0.0.0:23671, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=97, name=qtp1946791897-97 Acceptor2 SelectChannelConnector@0.0.0.0:32207, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=95, name=qtp1946791897-95 Acceptor0 SelectChannelConnector@0.0.0.0:32207, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=79, name=qtp1070454762-79 Acceptor2 SelectChannelConnector@0.0.0.0:37521, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=77, name=qtp1070454762-77 Acceptor0 SelectChannelConnector@0.0.0.0:37521, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=29, name=qtp399389739-29 Acceptor2 SelectChannelConnector@0.0.0.0:23671, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=98, name=qtp1946791897-98 Acceptor3 SelectChannelConnector@0.0.0.0:32207, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=80, name=qtp1070454762-80 Acceptor3 SelectChannelConnector@0.0.0.0:37521, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=78, name=qtp1070454762-78 Acceptor1 SelectChannelConnector@0.0.0.0:37521, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=45, name=qtp526921047-45 Acceptor0 SelectChannelConnector@0.0.0.0:54171, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=28, name=qtp399389739-28 Acceptor1 SelectChannelConnector@0.0.0.0:23671, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=48, name=qtp526921047-48 Acceptor3 SelectChannelConnector@0.0.0.0:54171, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=46, name=qtp526921047-46 Acceptor1 SelectChannelConnector@0.0.0.0:54171, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=27, name=qtp399389739-27 Acceptor0 SelectChannelConnector@0.0.0.0:23671, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([3AA65C7CB87FC8FA]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZk2Test

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=47, name=qtp526921047-47 Acceptor2 SelectChannelConnector@0.0.0.0:54171, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=96, name=qtp1946791897-96 Acceptor1 SelectChannelConnector@0.0.0.0:32207, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=30, name=qtp399389739-30 Acceptor3 SelectChannelConnector@0.0.0.0:23671, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=97, name=qtp1946791897-97 Acceptor2 SelectChannelConnector@0.0.0.0:32207, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=79, name=qtp1070454762-79 Acceptor2 SelectChannelConnector@0.0.0.0:37521, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=95, name=qtp1946791897-95 Acceptor0 SelectChannelConnector@0.0.0.0:32207, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=77, name=qtp1070454762-77 Acceptor0 SelectChannelConnector@0.0.0.0:37521, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=29, name=qtp399389739-29 Acceptor2 SelectChannelConnector@0.0.0.0:23671, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=98, name=qtp1946791897-98 Acceptor3 SelectChannelConnector@0.0.0.0:32207, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=80, name=qtp1070454762-80 Acceptor3 SelectChannelConnector@0.0.0.0:37521, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=78, name=qtp1070454762-78 Acceptor1 SelectChannelConnector@0.0.0.0:37521, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=45, name=qtp526921047-45 Acceptor0 SelectChannelConnector@0.0.0.0:54171, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=28, name=qtp399389739-28 Acceptor1 SelectChannelConnector@0.0.0.0:23671, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=48, name=qtp526921047-48 Acceptor3 SelectChannelConnector@0.0.0.0:54171, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=46, name=qtp526921047-46 Acceptor1 SelectChannelConnector@0.0.0.0:54171, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=27, name=qtp399389739-27 Acceptor0 SelectChannelConnector@0.0.0.0:23671, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=47, name=qtp526921047-47 Acceptor2 SelectChannelConnector@0.0.0.0:54171, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=96, name=qtp1946791897-96 Acceptor1 SelectChannelConnector@0.0.0.0:32207, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=30, name=qtp399389739-30 Acceptor3 SelectChannelConnector@0.0.0.0:23671, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=97, name=qtp1946791897-97 Acceptor2 SelectChannelConnector@0.0.0.0:32207, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=79, name=qtp1070454762-79 Acceptor2 SelectChannelConnector@0.0.0.0:37521, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=95, name=qtp1946791897-95 Acceptor0 SelectChannelConnector@0.0.0.0:32207, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=77, name=qtp1070454762-77 Acceptor0 SelectChannelConnector@0.0.0.0:37521, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=29, name=qtp399389739-29 Acceptor2 SelectChannelConnector@0.0.0.0:23671, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=98, name=qtp1946791897-98 Acceptor3 SelectChannelConnector@0.0.0.0:32207, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=80, name=qtp1070454762-80 Acceptor3 SelectChannelConnector@0.0.0.0:37521, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=78, name=qtp1070454762-78 Acceptor1 SelectChannelConnector@0.0.0.0:37521, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=45, name=qtp526921047-45 Acceptor0 SelectChannelConnector@0.0.0.0:54171, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=28, name=qtp399389739-28 Acceptor1 SelectChannelConnector@0.0.0.0:23671, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=48, name=qtp526921047-48 Acceptor3 SelectChannelConnector@0.0.0.0:54171, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=46, name=qtp526921047-46 Acceptor1 SelectChannelConnector@0.0.0.0:54171, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=27, name=qtp399389739-27 Acceptor0 SelectChannelConnector@0.0.0.0:23671, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([3AA65C7CB87FC8FA]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([3AA65C7CB87FC8FA]:0)


FAILED:  org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch

Error Message:
Shard still reported as live in zk

Stack Trace:
java.lang.AssertionError: Shard still reported as live in zk
	at __randomizedtesting.SeedInfo.seed([3AA65C7CB87FC8FA:BB40D264CF20A8C6]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitToSeeNotLive(AbstractFullDistribZkTestBase.java:1218)
	at org.apache.solr.cloud.BasicDistributedZk2Test.brindDownShardIndexSomeDocsAndRecover(BasicDistributedZk2Test.java:190)
	at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:83)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:695)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)


FAILED:  org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testDistribSearch

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([3AA65C7CB87FC8FA]:0)




Build Log:
[...truncated 8309 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
[junit4:junit4]   2> 233 T11 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1352168085568
[junit4:junit4]   2> 248 T11 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 347 T12 oaz.Environment.logEnv Server environment:zookeeper.version=3.3.6-1366786, built on 07/29/2012 06:22 GMT
[junit4:junit4]   2> 349 T12 oaz.Environment.logEnv Server environment:host.name=lucene.zones.apache.org
[junit4:junit4]   2> 349 T12 oaz.Environment.logEnv Server environment:java.version=1.7.0_09
[junit4:junit4]   2> 350 T12 oaz.Environment.logEnv Server environment:java.vendor=Oracle Corporation
[junit4:junit4]   2> 350 T12 oaz.Environment.logEnv Server environment:java.home=/usr/local/openjdk7/jre
[junit4:junit4]   2> 351 T12 oaz.Environment.logEnv Server environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/common/lucene-analyzers-common-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/highlighter/lucene-highlighter-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/memory/lucene-memory-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/misc/lucene-misc-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/spatial/lucene-spatial-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/suggest/lucene-suggest-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/grouping/lucene-grouping-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queries/lucene-queries-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queryparser/lucene-queryparser-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/easymock-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/guava-r05.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/log4j-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/zookeeper-3.3.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-continuation-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-deploy-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-http-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-io-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-jmx-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-security-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-server-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-servlet-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-util-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-webapp-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-xml-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-javamail.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jdepend.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.4.jar
[junit4:junit4]   2> 355 T12 oaz.Environment.logEnv Server environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
[junit4:junit4]   2> 356 T12 oaz.Environment.logEnv Server environment:java.io.tmpdir=.
[junit4:junit4]   2> 357 T12 oaz.Environment.logEnv Server environment:java.compiler=<NA>
[junit4:junit4]   2> 357 T12 oaz.Environment.logEnv Server environment:os.name=FreeBSD
[junit4:junit4]   2> 358 T12 oaz.Environment.logEnv Server environment:os.arch=amd64
[junit4:junit4]   2> 358 T12 oaz.Environment.logEnv Server environment:os.version=9.0-RELEASE
[junit4:junit4]   2> 359 T12 oaz.Environment.logEnv Server environment:user.name=hudson
[junit4:junit4]   2> 359 T12 oaz.Environment.logEnv Server environment:user.home=/home/hudson
[junit4:junit4]   2> 360 T12 oaz.Environment.logEnv Server environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 397 T12 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 428 T12 oazs.NIOServerCnxn$Factory.<init> binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 457 T11 oasc.ZkTestServer.run start zk server on port:33140
[junit4:junit4]   2> 490 T12 oazsp.FileTxnSnapLog.save Snapshotting: 0
[junit4:junit4]   2> 523 T11 oaz.Environment.logEnv Client environment:zookeeper.version=3.3.6-1366786, built on 07/29/2012 06:22 GMT
[junit4:junit4]   2> 524 T11 oaz.Environment.logEnv Client environment:host.name=lucene.zones.apache.org
[junit4:junit4]   2> 525 T11 oaz.Environment.logEnv Client environment:java.version=1.7.0_09
[junit4:junit4]   2> 526 T11 oaz.Environment.logEnv Client environment:java.vendor=Oracle Corporation
[junit4:junit4]   2> 526 T11 oaz.Environment.logEnv Client environment:java.home=/usr/local/openjdk7/jre
[junit4:junit4]   2> 527 T11 oaz.Environment.logEnv Client environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/common/lucene-analyzers-common-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/highlighter/lucene-highlighter-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/memory/lucene-memory-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/misc/lucene-misc-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/spatial/lucene-spatial-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/suggest/lucene-suggest-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/grouping/lucene-grouping-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queries/lucene-queries-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queryparser/lucene-queryparser-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/easymock-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/guava-r05.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/log4j-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/zookeeper-3.3.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-continuation-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-deploy-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-http-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-io-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-jmx-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-security-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-server-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-servlet-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-util-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-webapp-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-xml-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-javamail.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jdepend.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.4.jar
[junit4:junit4]   2> 530 T11 oaz.Environment.logEnv Client environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
[junit4:junit4]   2> 530 T11 oaz.Environment.logEnv Client environment:java.io.tmpdir=.
[junit4:junit4]   2> 531 T11 oaz.Environment.logEnv Client environment:java.compiler=<NA>
[junit4:junit4]   2> 532 T11 oaz.Environment.logEnv Client environment:os.name=FreeBSD
[junit4:junit4]   2> 532 T11 oaz.Environment.logEnv Client environment:os.arch=amd64
[junit4:junit4]   2> 533 T11 oaz.Environment.logEnv Client environment:os.version=9.0-RELEASE
[junit4:junit4]   2> 534 T11 oaz.Environment.logEnv Client environment:user.name=hudson
[junit4:junit4]   2> 534 T11 oaz.Environment.logEnv Client environment:user.home=/home/hudson
[junit4:junit4]   2> 535 T11 oaz.Environment.logEnv Client environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 539 T11 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33140 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@3fafe567
[junit4:junit4]   2> 581 T16 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:33140
[junit4:junit4]   2> 582 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 587 T16 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33140, initiating session
[junit4:junit4]   2> 588 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:15787
[junit4:junit4]   2> 605 T13 oazs.NIOServerCnxn.doIO WARNING Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
[junit4:junit4]   2> 606 T13 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:15787 (no session established for client)
[junit4:junit4]   2> 607 T16 oaz.ClientCnxn$SendThread.run Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
[junit4:junit4]   2> 1878 T16 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost/127.0.0.1:33140
[junit4:junit4]   2> 1879 T16 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33140, initiating session
[junit4:junit4]   2> 1879 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:35504
[junit4:junit4]   2> 1881 T13 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:35504
[junit4:junit4]   2> 1900 T15 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 1984 T15 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad37f995c0000 with negotiated timeout 10000 for client /140.211.11.196:35504
[junit4:junit4]   2> 1984 T16 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:33140, sessionid = 0x13ad37f995c0000, negotiated timeout = 10000
[junit4:junit4]   2> 1990 T18 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3fafe567 name:ZooKeeperConnection Watcher:127.0.0.1:33140 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1991 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1996 T11 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 2116 T17 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x13ad37f995c0000
[junit4:junit4]   2> 2122 T18 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 2122 T11 oaz.ZooKeeper.close Session: 0x13ad37f995c0000 closed
[junit4:junit4]   2> 2123 T13 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:35504 which had sessionid 0x13ad37f995c0000
[junit4:junit4]   2> 2124 T11 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33140/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@3909f81a
[junit4:junit4]   2> 2126 T19 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:33140
[junit4:junit4]   2> 2126 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2129 T19 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33140, initiating session
[junit4:junit4]   2> 2129 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:42943
[junit4:junit4]   2> 2130 T13 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:42943
[junit4:junit4]   2> 2133 T15 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad37f995c0001 with negotiated timeout 10000 for client /140.211.11.196:42943
[junit4:junit4]   2> 2133 T19 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:33140, sessionid = 0x13ad37f995c0001, negotiated timeout = 10000
[junit4:junit4]   2> 2135 T20 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3909f81a name:ZooKeeperConnection Watcher:127.0.0.1:33140/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2135 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2177 T11 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 2184 T11 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 2190 T11 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 2223 T11 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 2229 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2288 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2305 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 2307 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 2416 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2419 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2462 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 2464 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 2469 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 2471 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 2476 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 2478 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 2483 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 2485 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 2490 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 2492 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 2497 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 2499 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 2504 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 2506 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 2511 T17 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x13ad37f995c0001
[junit4:junit4]   2> 2513 T20 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 2513 T13 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:42943 which had sessionid 0x13ad37f995c0001
[junit4:junit4]   2> 2513 T11 oaz.ZooKeeper.close Session: 0x13ad37f995c0001 closed
[junit4:junit4]   2> 3178 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 3602 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:23671
[junit4:junit4]   2> 3832 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3833 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 3833 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4432 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 4436 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4437 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 4468 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 4473 T11 oasc.CoreContainer.<init> New CoreContainer 873441026
[junit4:junit4]   2> 4474 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4475 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4562 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4593 T11 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:33140/solr
[junit4:junit4]   2> 4631 T11 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33140/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@1fcebee
[junit4:junit4]   2> 4633 T35 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:33140
[junit4:junit4]   2> 4633 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4635 T35 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33140, initiating session
[junit4:junit4]   2> 4636 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:51841
[junit4:junit4]   2> 4637 T13 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:51841
[junit4:junit4]   2> 4639 T15 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad37f995c0002 with negotiated timeout 5000 for client /140.211.11.196:51841
[junit4:junit4]   2> 4639 T35 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:33140, sessionid = 0x13ad37f995c0002, negotiated timeout = 5000
[junit4:junit4]   2> 4640 T36 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fcebee name:ZooKeeperConnection Watcher:127.0.0.1:33140/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4641 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4658 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0002 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4668 T11 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 4671 T11 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:23671_solr
[junit4:junit4]   2> 4677 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0002 type:delete cxid:0x7 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:23671_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:23671_solr
[junit4:junit4]   2> 4678 T11 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:23671_solr
[junit4:junit4]   2> 4693 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 4694 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4695 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 4695 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4696 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4697 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4697 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4698 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4698 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4746 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 5206 T11 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 5229 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0002 type:delete cxid:0x15 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 5230 T11 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 5234 T11 oasc.Overseer.start Overseer (id=88615687674396674-127.0.0.1:23671_solr-n_0000000000) starting
[junit4:junit4]   2> 5235 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0002 type:create cxid:0x1a zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5251 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0002 type:create cxid:0x1b zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5252 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0002 type:create cxid:0x1c zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5263 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0002 type:create cxid:0x1d zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5265 T38 oasc.OverseerCollectionProcessor.run Process current queue of collection messages
[junit4:junit4]   2> 5265 T11 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 5269 T11 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 5293 T37 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 5347 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 5348 T11 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 5349 T11 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 5350 T11 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 5352 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 5354 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 5355 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 5619 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 5831 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 5961 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 6064 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 7765 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 7793 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 7797 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 8184 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8206 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8213 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8220 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8306 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/control/data/
[junit4:junit4]   2> 8315 T11 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ac91010
[junit4:junit4]   2> 8537 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/control/data/index/
[junit4:junit4]   2> 8539 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 8649 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/control/data/index forceNew:false
[junit4:junit4]   2> 9087 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@55958273; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 9088 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 9134 T11 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 10356 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 10357 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 10357 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 10368 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 10511 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 10526 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 10531 T11 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 10568 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 10593 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 10669 T11 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 10671 T11 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 10678 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 10679 T11 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 10680 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 10681 T11 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 10721 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 10722 T11 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10723 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10724 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10725 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10726 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10728 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10729 T11 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10755 T11 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 10781 T11 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 10965 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 11000 T11 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 11009 T11 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 11084 T11 oass.SolrIndexSearcher.<init> Opening Searcher@7377dfc2 main
[junit4:junit4]   2> 11212 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 11213 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 11214 T11 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 11387 T11 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 11807 T39 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7377dfc2 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 11826 T11 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11827 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0002 type:create cxid:0x77 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 11900 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 11901 T11 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:23671/solr shardId:control_shard
[junit4:junit4]   2> 11917 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 11940 T11 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/control_shard/election
[junit4:junit4]   2> 12016 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0002 type:delete cxid:0x86 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 12046 T11 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 12047 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0002 type:create cxid:0x87 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12327 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12328 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"control_shard",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:23671_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:23671/solr"}
[junit4:junit4]   2> 12344 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0002 type:create cxid:0x8e zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 12395 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (1)
[junit4:junit4]   2> 12558 T11 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 12558 T11 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 12559 T11 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:23671/solr/collection1/
[junit4:junit4]   2> 12560 T11 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 12560 T11 oasc.SyncStrategy.syncToMe http://127.0.0.1:23671/solr/collection1/ has no replicas
[junit4:junit4]   2> 12561 T11 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:23671/solr/collection1/
[junit4:junit4]   2> 12561 T11 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/control_shard
[junit4:junit4]   2> 12576 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0002 type:create cxid:0xac zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12903 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12917 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (1)
[junit4:junit4]   2> 12943 T11 oasc.ZkController.register We are http://127.0.0.1:23671/solr/collection1/ and leader is http://127.0.0.1:23671/solr/collection1/
[junit4:junit4]   2> 12946 T11 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:23671/solr
[junit4:junit4]   2> 12947 T11 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 12947 T11 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 12950 T11 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12951 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 12952 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 13000 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 13141 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 13144 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:54171
[junit4:junit4]   2> 13145 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 13145 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352168098346
[junit4:junit4]   2> 13146 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352168098346/'
[junit4:junit4]   2> 13178 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 13178 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 13179 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352168098346
[junit4:junit4]   2> 13179 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352168098346/solr.xml
[junit4:junit4]   2> 13180 T11 oasc.CoreContainer.<init> New CoreContainer 230897924
[junit4:junit4]   2> 13181 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352168098346/'
[junit4:junit4]   2> 13181 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352168098346/'
[junit4:junit4]   2> 13209 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 13224 T11 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:33140/solr
[junit4:junit4]   2> 13225 T11 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33140/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@70745955
[junit4:junit4]   2> 13226 T49 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:33140
[junit4:junit4]   2> 13226 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 13228 T49 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33140, initiating session
[junit4:junit4]   2> 13228 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:45948
[junit4:junit4]   2> 13229 T13 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:45948
[junit4:junit4]   2> 13231 T15 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad37f995c0003 with negotiated timeout 5000 for client /140.211.11.196:45948
[junit4:junit4]   2> 13231 T49 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:33140, sessionid = 0x13ad37f995c0003, negotiated timeout = 5000
[junit4:junit4]   2> 13232 T50 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@70745955 name:ZooKeeperConnection Watcher:127.0.0.1:33140/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 13232 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 13233 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0003 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13241 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0003 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13243 T11 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54171_solr
[junit4:junit4]   2> 13243 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0003 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:54171_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:54171_solr
[junit4:junit4]   2> 13245 T11 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:54171_solr
[junit4:junit4]   2> 13248 T36 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 13249 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 13249 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 13250 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 13250 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 13251 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 13251 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 13252 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 13252 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 13253 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 13253 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 13271 T11 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 13278 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352168098346/collection1
[junit4:junit4]   2> 13278 T11 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 13279 T11 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 13280 T11 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 13282 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352168098346/collection1/'
[junit4:junit4]   2> 13283 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty1-1352168098346/collection1/lib/README' to classloader
[junit4:junit4]   2> 13284 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty1-1352168098346/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 13394 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 13422 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13424 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"control_shard",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:23671_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:23671/solr"}
[junit4:junit4]   2> 13430 T50 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 13430 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 13489 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 13591 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 13662 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 14425 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 14439 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 14443 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 14470 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 14476 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 14482 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 14486 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 14489 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352168098346/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty1/
[junit4:junit4]   2> 14490 T11 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ac91010
[junit4:junit4]   2> 14491 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty1/index/
[junit4:junit4]   2> 14492 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 14493 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty1/index forceNew:false
[junit4:junit4]   2> 14499 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@49a52b21; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 14500 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 14501 T11 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 14505 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 14505 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 14506 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 14506 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 14507 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 14508 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 14509 T11 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 14510 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 14512 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 14513 T11 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 14515 T11 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 14516 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 14517 T11 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 14517 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 14518 T11 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 14520 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 14521 T11 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14523 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14524 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14525 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14527 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14528 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14530 T11 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14531 T11 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 14533 T11 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 14542 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 14546 T11 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 14548 T11 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 14553 T11 oass.SolrIndexSearcher.<init> Opening Searcher@66c0bc57 main
[junit4:junit4]   2> 14554 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 14554 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 14556 T11 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 14563 T11 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 14629 T55 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@66c0bc57 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 14640 T11 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14947 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14948 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:54171_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:54171/solr"}
[junit4:junit4]   2> 14949 T37 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 14960 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 14960 T50 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 15144 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 15144 T11 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:54171/solr shardId:shard1
[junit4:junit4]   2> 15146 T11 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 15170 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0003 type:delete cxid:0x43 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 15178 T11 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 15179 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0003 type:create cxid:0x44 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15188 T11 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 15189 T11 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 15189 T11 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:54171/solr/collection1/
[junit4:junit4]   2> 15190 T11 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 15190 T11 oasc.SyncStrategy.syncToMe http://127.0.0.1:54171/solr/collection1/ has no replicas
[junit4:junit4]   2> 15191 T11 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:54171/solr/collection1/
[junit4:junit4]   2> 15191 T11 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 15205 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0003 type:create cxid:0x4e zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15472 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15490 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 15491 T50 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 15513 T11 oasc.ZkController.register We are http://127.0.0.1:54171/solr/collection1/ and leader is http://127.0.0.1:54171/solr/collection1/
[junit4:junit4]   2> 15513 T11 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:54171/solr
[junit4:junit4]   2> 15514 T11 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 15514 T11 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15517 T11 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15518 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 15519 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 15519 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 15656 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 15659 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:54465
[junit4:junit4]   2> 15660 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 15661 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352168100860
[junit4:junit4]   2> 15661 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352168100860/'
[junit4:junit4]   2> 15694 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 15695 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 15696 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352168100860
[junit4:junit4]   2> 15696 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352168100860/solr.xml
[junit4:junit4]   2> 15697 T11 oasc.CoreContainer.<init> New CoreContainer 872227382
[junit4:junit4]   2> 15697 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352168100860/'
[junit4:junit4]   2> 15698 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352168100860/'
[junit4:junit4]   2> 15728 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 15745 T11 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:33140/solr
[junit4:junit4]   2> 15746 T11 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33140/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@6b3e3c75
[junit4:junit4]   2> 15747 T65 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:33140
[junit4:junit4]   2> 15747 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 15749 T65 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33140, initiating session
[junit4:junit4]   2> 15749 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:49378
[junit4:junit4]   2> 15750 T13 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:49378
[junit4:junit4]   2> 15751 T15 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad37f995c0004 with negotiated timeout 5000 for client /140.211.11.196:49378
[junit4:junit4]   2> 15751 T65 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:33140, sessionid = 0x13ad37f995c0004, negotiated timeout = 5000
[junit4:junit4]   2> 15752 T66 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6b3e3c75 name:ZooKeeperConnection Watcher:127.0.0.1:33140/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 15753 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 15754 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0004 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15755 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0004 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15757 T11 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54465_solr
[junit4:junit4]   2> 15758 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0004 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:54465_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:54465_solr
[junit4:junit4]   2> 15764 T11 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:54465_solr
[junit4:junit4]   2> 15768 T36 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 15768 T50 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 15769 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 15770 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 15770 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 15771 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 15771 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 15772 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 15772 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 15773 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 15773 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 15774 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 15791 T11 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 15799 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352168100860/collection1
[junit4:junit4]   2> 15800 T11 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 15801 T11 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 15802 T11 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 15804 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352168100860/collection1/'
[junit4:junit4]   2> 15806 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty2-1352168100860/collection1/lib/README' to classloader
[junit4:junit4]   2> 15808 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty2-1352168100860/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 15898 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 15961 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 16006 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16008 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:54171_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:54171/solr"}
[junit4:junit4]   2> 16021 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 16021 T50 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 16062 T66 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 16062 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 16103 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 16665 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 16679 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 16683 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 16731 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 16744 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 16755 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 16761 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 16767 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352168100860/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty2/
[junit4:junit4]   2> 16768 T11 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ac91010
[junit4:junit4]   2> 16772 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty2/index/
[junit4:junit4]   2> 16773 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 16775 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty2/index forceNew:false
[junit4:junit4]   2> 16790 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@255b252c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 16792 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 16794 T11 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 16801 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 16803 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 16804 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 16807 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 16809 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 16811 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 16813 T11 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 16816 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 16818 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 16821 T11 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 16824 T11 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 16826 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 16829 T11 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 16830 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 16832 T11 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 16835 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 16838 T11 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16841 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16843 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16846 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16849 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16852 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16855 T11 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16857 T11 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 16860 T11 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 16882 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 16892 T11 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 16895 T11 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 16905 T11 oass.SolrIndexSearcher.<init> Opening Searcher@327b9698 main
[junit4:junit4]   2> 16909 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 16910 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 16913 T11 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 16973 T11 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 17116 T71 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@327b9698 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 17146 T11 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 17534 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17536 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:54465_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:54465/solr"}
[junit4:junit4]   2> 17556 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 17557 T66 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 17557 T50 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 17650 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 17651 T11 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:54465/solr shardId:shard2
[junit4:junit4]   2> 17652 T11 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 17670 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0004 type:delete cxid:0x42 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 17671 T11 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 17672 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0004 type:create cxid:0x43 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 17681 T11 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 17682 T11 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 17682 T11 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:54465/solr/collection1/
[junit4:junit4]   2> 17683 T11 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 17683 T11 oasc.SyncStrategy.syncToMe http://127.0.0.1:54465/solr/collection1/ has no replicas
[junit4:junit4]   2> 17684 T11 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:54465/solr/collection1/
[junit4:junit4]   2> 17684 T11 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 17690 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0004 type:create cxid:0x4c zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 18068 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18095 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 18095 T50 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 18095 T66 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 18101 T11 oasc.ZkController.register We are http://127.0.0.1:54465/solr/collection1/ and leader is http://127.0.0.1:54465/solr/collection1/
[junit4:junit4]   2> 18101 T11 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:54465/solr
[junit4:junit4]   2> 18102 T11 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 18102 T11 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 18105 T11 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18107 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 18107 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 18108 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 18232 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 18234 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:37521
[junit4:junit4]   2> 18236 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 18236 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352168103448
[junit4:junit4]   2> 18237 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352168103448/'
[junit4:junit4]   2> 18266 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 18267 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 18268 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352168103448
[junit4:junit4]   2> 18268 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352168103448/solr.xml
[junit4:junit4]   2> 18268 T11 oasc.CoreContainer.<init> New CoreContainer 332661287
[junit4:junit4]   2> 18269 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352168103448/'
[junit4:junit4]   2> 18270 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352168103448/'
[junit4:junit4]   2> 18295 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 18309 T11 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:33140/solr
[junit4:junit4]   2> 18310 T11 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33140/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@4febd0d
[junit4:junit4]   2> 18311 T81 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:33140
[junit4:junit4]   2> 18311 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 18312 T81 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33140, initiating session
[junit4:junit4]   2> 18313 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:55601
[junit4:junit4]   2> 18313 T13 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:55601
[junit4:junit4]   2> 18316 T15 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad37f995c0005 with negotiated timeout 5000 for client /140.211.11.196:55601
[junit4:junit4]   2> 18316 T81 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:33140, sessionid = 0x13ad37f995c0005, negotiated timeout = 5000
[junit4:junit4]   2> 18316 T82 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4febd0d name:ZooKeeperConnection Watcher:127.0.0.1:33140/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 18317 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 18318 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0005 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 18320 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0005 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 18322 T11 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37521_solr
[junit4:junit4]   2> 18322 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0005 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:37521_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:37521_solr
[junit4:junit4]   2> 18330 T11 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37521_solr
[junit4:junit4]   2> 18334 T50 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 18334 T36 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 18334 T66 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 18337 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 18337 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 18338 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 18338 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 18338 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 18339 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 18339 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 18340 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 18340 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 18341 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 18357 T11 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 18363 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352168103448/collection1
[junit4:junit4]   2> 18364 T11 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 18365 T11 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 18365 T11 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 18367 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352168103448/collection1/'
[junit4:junit4]   2> 18368 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty3-1352168103448/collection1/lib/README' to classloader
[junit4:junit4]   2> 18369 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty3-1352168103448/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 18425 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 18485 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 18587 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 18603 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18605 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:54465_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:54465/solr"}
[junit4:junit4]   2> 18619 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 18619 T50 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 18619 T66 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 18619 T82 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 18621 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 19140 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 19153 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 19156 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 19183 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 19189 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 19193 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 19196 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 19199 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352168103448/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty3/
[junit4:junit4]   2> 19200 T11 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ac91010
[junit4:junit4]   2> 19201 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty3/index/
[junit4:junit4]   2> 19202 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 19203 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty3/index forceNew:false
[junit4:junit4]   2> 19245 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@53323184; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 19246 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 19248 T11 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 19252 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 19253 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 19254 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 19255 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 19258 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 19258 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 19259 T11 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 19261 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 19262 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 19264 T11 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 19265 T11 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 19267 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 19269 T11 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 19270 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 19271 T11 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 19274 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 19276 T11 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19277 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19279 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19280 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19290 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19291 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19292 T11 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19293 T11 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 19294 T11 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 19303 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 19307 T11 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 19308 T11 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 19313 T11 oass.SolrIndexSearcher.<init> Opening Searcher@53d4287 main
[junit4:junit4]   2> 19314 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 19314 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 19315 T11 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 19323 T11 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 19387 T87 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@53d4287 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 19394 T11 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 19629 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19631 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:37521_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:37521/solr"}
[junit4:junit4]   2> 19645 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 19645 T50 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 19645 T66 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 19645 T82 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 19898 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 19898 T11 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:37521/solr shardId:shard1
[junit4:junit4]   2> 19903 T11 oasc.ZkController.register We are http://127.0.0.1:37521/solr/collection1/ and leader is http://127.0.0.1:54171/solr/collection1/
[junit4:junit4]   2> 19904 T11 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:37521/solr
[junit4:junit4]   2> 19904 T11 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 19905 T11 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> 19912 T11 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19913 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 19913 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 19914 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C0 name=collection1 org.apache.solr.core.SolrCore@5636f88c url=http://127.0.0.1:37521/solr/collection1 node=127.0.0.1:37521_solr
[junit4:junit4]   2> 19921 T88 C0 P37521 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 19927 T88 C0 P37521 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 19927 T88 C0 P37521 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 19935 T88 C0 P37521 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:54171/solr/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 19948 T88 C0 P37521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 19976 T88 C0 P37521 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:37521/solr START replicas=[http://127.0.0.1:54171/solr/collection1/] nUpdates=100
[junit4:junit4]   2> 20000 T88 C0 P37521 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 20001 T88 C0 P37521 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 20002 T88 C0 P37521 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 20002 T88 C0 P37521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 20123 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 20127 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:32207
[junit4:junit4]   2> 20129 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 20130 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty4-1352168105256
[junit4:junit4]   2> 20131 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty4-1352168105256/'
[junit4:junit4]   2> 20151 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20152 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:37521_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:37521/solr"}
[junit4:junit4]   2> 20159 T50 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 20159 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 20159 T82 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 20159 T66 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 20222 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 20223 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 20224 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty4-1352168105256
[junit4:junit4]   2> 20225 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty4-1352168105256/solr.xml
[junit4:junit4]   2> 20226 T11 oasc.CoreContainer.<init> New CoreContainer 349497597
[junit4:junit4]   2> 20228 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty4-1352168105256/'
[junit4:junit4]   2> 20229 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty4-1352168105256/'
[junit4:junit4]   2> 20289 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 20314 T11 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:33140/solr
[junit4:junit4]   2> 20315 T11 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33140/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@84373ce
[junit4:junit4]   2> 20316 T99 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:33140
[junit4:junit4]   2> 20316 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 20319 T99 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33140, initiating session
[junit4:junit4]   2> 20319 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:49628
[junit4:junit4]   2> 20320 T13 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:49628
[junit4:junit4]   2> 20322 T15 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad37f995c0006 with negotiated timeout 5000 for client /140.211.11.196:49628
[junit4:junit4]   2> 20322 T99 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:33140, sessionid = 0x13ad37f995c0006, negotiated timeout = 5000
[junit4:junit4]   2> 20323 T100 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@84373ce name:ZooKeeperConnection Watcher:127.0.0.1:33140/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 20323 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 20324 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0006 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 20326 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0006 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 20328 T11 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32207_solr
[junit4:junit4]   2> 20329 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0006 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:32207_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:32207_solr
[junit4:junit4]   2> 20330 T11 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:32207_solr
[junit4:junit4]   2> 20343 T50 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 20343 T82 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 20343 T66 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 20343 T36 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 20344 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 20345 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 20346 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 20346 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 20347 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 20348 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 20348 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 20349 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 20350 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 20351 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 20373 T11 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 20385 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty4-1352168105256/collection1
[junit4:junit4]   2> 20386 T11 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 20387 T11 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 20388 T11 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 20390 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty4-1352168105256/collection1/'
[junit4:junit4]   2> 20392 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty4-1352168105256/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 20394 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty4-1352168105256/collection1/lib/README' to classloader
[junit4:junit4]   2> 20531 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 20623 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 20725 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 20810 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> ASYNC  NEW_CORE C1 name=collection1 org.apache.solr.core.SolrCore@7edeaabc url=http://127.0.0.1:54171/solr/collection1 node=127.0.0.1:54171_solr
[junit4:junit4]   2> 21123 T52 C1 P54171 REQ /get {distrib=false&qt=/get&wt=javabin&version=2&getVersions=100} status=0 QTime=41 
[junit4:junit4]   2> 21824 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 21844 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 21849 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 21878 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 21890 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 21895 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 21908 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 21911 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty4-1352168105256/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty4/
[junit4:junit4]   2> 21911 T11 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ac91010
[junit4:junit4]   2> 21913 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty4/index/
[junit4:junit4]   2> 21917 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 21918 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty4/index forceNew:false
[junit4:junit4]   2> 21970 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3bc02228; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 21971 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 21972 T11 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 21977 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 21977 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 21978 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 21978 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 21979 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 21980 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 21980 T11 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 21981 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 21982 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 21983 T11 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 21984 T11 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 21985 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 21986 T11 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 21986 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 21987 T11 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 21988 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 21989 T11 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 21990 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 21991 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 21992 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 21992 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 21993 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 21994 T11 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 21995 T11 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 21996 T11 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 22004 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 22008 T11 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 22010 T11 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 22013 T11 oass.SolrIndexSearcher.<init> Opening Searcher@9c89957 main
[junit4:junit4]   2> 22015 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 22015 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 22017 T11 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 22035 T11 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 22108 T104 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@9c89957 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 22117 T11 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 22179 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 22180 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:32207_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:32207/solr"}
[junit4:junit4]   2> 22193 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (5)
[junit4:junit4]   2> 22193 T66 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (5)
[junit4:junit4]   2> 22193 T100 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (5)
[junit4:junit4]   2> 22193 T50 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (5)
[junit4:junit4]   2> 22193 T82 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (5)
[junit4:junit4]   2> 22620 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 22620 T11 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:32207/solr shardId:shard2
[junit4:junit4]   2> 22624 T11 oasc.ZkController.register We are http://127.0.0.1:32207/solr/collection1/ and leader is http://127.0.0.1:54465/solr/collection1/
[junit4:junit4]   2> 22624 T11 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:32207/solr
[junit4:junit4]   2> 22625 T11 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 22625 T11 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C2 name=collection1 org.apache.solr.core.SolrCore@ce14ed3 url=http://127.0.0.1:32207/solr/collection1 node=127.0.0.1:32207_solr
[junit4:junit4]   2> 22626 T105 C2 P32207 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 22627 T11 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 22627 T105 C2 P32207 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 22627 T105 C2 P32207 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 22628 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 22629 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 22629 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 22631 T105 C2 P32207 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:54465/solr/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 22631 T105 C2 P32207 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:32207/solr START replicas=[http://127.0.0.1:54465/solr/collection1/] nUpdates=100
[junit4:junit4]   2> 22632 T105 C2 P32207 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 22632 T105 C2 P32207 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 22633 T105 C2 P32207 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 22633 T105 C2 P32207 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C3 name=collection1 org.apache.solr.core.SolrCore@7fe69211 url=http://127.0.0.1:54465/solr/collection1 node=127.0.0.1:54465_solr
[junit4:junit4]   2> 22649 T69 C3 P54465 REQ /get {distrib=false&qt=/get&wt=javabin&version=2&getVersions=100} status=0 QTime=1 
[junit4:junit4]   2> 22657 T11 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33140/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@4f77037a
[junit4:junit4]   2> 22658 T107 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:33140
[junit4:junit4]   2> 22658 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 22659 T107 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33140, initiating session
[junit4:junit4]   2> 22659 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:29328
[junit4:junit4]   2> 22660 T13 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:29328
[junit4:junit4]   2> 22661 T15 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad37f995c0007 with negotiated timeout 10000 for client /140.211.11.196:29328
[junit4:junit4]   2> 22662 T107 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:33140, sessionid = 0x13ad37f995c0007, negotiated timeout = 10000
[junit4:junit4]   2> 22662 T108 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4f77037a name:ZooKeeperConnection Watcher:127.0.0.1:33140/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 22662 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 22663 T11 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 22667 T11 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 22668 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 22678 T11 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 22698 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 22700 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:32207_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:32207/solr"}
[junit4:junit4]   2> 22711 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (5)
[junit4:junit4]   2> 22711 T50 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (5)
[junit4:junit4]   2> 22711 T100 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (5)
[junit4:junit4]   2> 22711 T82 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (5)
[junit4:junit4]   2> 22711 T66 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (5)
[junit4:junit4]   2> 22711 T108 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (5)
[junit4:junit4]   2> ASYNC  NEW_CORE C4 name=collection1 org.apache.solr.core.SolrCore@2bc1e78c url=http://127.0.0.1:23671/solr/collection1 node=127.0.0.1:23671_solr
[junit4:junit4]   2> 22810 T32 C4 P23671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16
[junit4:junit4]   2>  C0_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:37521_solr, base_url=http://127.0.0.1:37521/solr}
[junit4:junit4]   2> 23214 T88 C0 P37521 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 23214 T88 C0 P37521 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 23215 T88 C0 P37521 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:54171/solr/collection1/. core=collection1
[junit4:junit4]   2> 23215 T88 C0 P37521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 23260 T53 C1 P54171 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 23268 T53 C1 P54171 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@49a52b21; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 23269 T53 C1 P54171 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 23271 T53 C1 P54171 oass.SolrIndexSearcher.<init> Opening Searcher@58a386c4 realtime
[junit4:junit4]   2> 23271 T53 C1 P54171 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 23272 T53 C1 P54171 /update {waitSearcher=true&openSearcher=false&commit=true&commit_end_point=true&wt=javabin&softCommit=false&version=2} {commit=} 0 32
[junit4:junit4]   2> 23338 T88 C0 P37521 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 23338 T88 C0 P37521 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4]   2> 23366 T88 C0 P37521 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 2

[...truncated too long message...]

erver.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  21) Thread[id=112, name=TEST-FullSolrCloudDistribCmdsTest.testDistribSearch-seed#[3AA65C7CB87FC8FA]-SendThread(localhost:38567), state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4:junit4]   2> 	        at sun.nio.ch.IOUtil.drain(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.updateSelectedKeys(KQueueSelectorImpl.java:167)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:108)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1131)
[junit4:junit4]   2> 	  22) Thread[id=60, name=qtp2054141666-60 Selector1, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.Timeout.expired(Timeout.java:97)
[junit4:junit4]   2> 	        at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:686)
[junit4:junit4]   2> 	        at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:285)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  23) Thread[id=30, name=qtp504163721-30 Acceptor1 SelectChannelConnector@0.0.0.0:24570, state=BLOCKED, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  24) Thread[id=106, name=searcherExecutor-41-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  25) Thread[id=66, name=qtp2054141666-66 Acceptor3 SelectChannelConnector@0.0.0.0:42265, state=BLOCKED, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  26) Thread[id=62, name=qtp2054141666-62 Selector3, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueArrayWrapper.interrupt(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueArrayWrapper.interrupt(KQueueArrayWrapper.java:164)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.wakeup(KQueueSelectorImpl.java:245)
[junit4:junit4]   2> 	        at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:210)
[junit4:junit4]   2> 	        at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:216)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:102)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4]   2> 	        at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:564)
[junit4:junit4]   2> 	        at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:285)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  27) Thread[id=95, name=qtp1627197352-95 Selector3, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:159)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.selectNow(SelectorImpl.java:106)
[junit4:junit4]   2> 	        at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:531)
[junit4:junit4]   2> 	        at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:285)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  28) Thread[id=99, name=qtp1627197352-99 Acceptor3 SelectChannelConnector@0.0.0.0:65270, state=BLOCKED, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Appending, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=fi, timezone=Asia/Irkutsk
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=36,free=83267720,total=135266304
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestDocSet, FullSolrCloudDistribCmdsTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=FullSolrCloudDistribCmdsTest -Dtests.seed=3AA65C7CB87FC8FA -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=fi -Dtests.timezone=Asia/Irkutsk -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] ERROR   0.00s J1 | FullSolrCloudDistribCmdsTest (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([3AA65C7CB87FC8FA]:0)
[junit4:junit4] Completed on J1 in 7223.97s, 1 test, 2 errors <<< FAILURES!

[...truncated 184 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:335: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/module-build.xml:61: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:1163: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:827: There were test failures: 233 suites, 965 tests, 3 suite-level errors, 1 error, 1 failure, 878 ignored

Total time: 138 minutes 1 second
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure

[JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 611 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.

Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/611/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.TestDistributedSearch

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([43BDC8EDB8C9CCE8]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.update.SolrCmdDistributorTest

Error Message:
20 threads leaked from SUITE scope at org.apache.solr.update.SolrCmdDistributorTest:     1) Thread[id=61, name=qtp1984873408-61 Acceptor1 SelectChannelConnector@0.0.0.0:54029, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=48, name=qtp1173231242-48 Acceptor2 SelectChannelConnector@0.0.0.0:21937, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=33, name=qtp1271263678-33 Acceptor1 SelectChannelConnector@0.0.0.0:31121, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=60, name=qtp1984873408-60 Acceptor0 SelectChannelConnector@0.0.0.0:54029, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=74, name=qtp1189333092-74 Acceptor0 SelectChannelConnector@0.0.0.0:40308, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=76, name=qtp1189333092-76 Acceptor2 SelectChannelConnector@0.0.0.0:40308, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=77, name=qtp1189333092-77 Acceptor3 SelectChannelConnector@0.0.0.0:40308, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=75, name=qtp1189333092-75 Acceptor1 SelectChannelConnector@0.0.0.0:40308, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=46, name=qtp1173231242-46 Acceptor0 SelectChannelConnector@0.0.0.0:21937, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=32, name=qtp1271263678-32 Acceptor0 SelectChannelConnector@0.0.0.0:31121, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=15, name=qtp191968531-15 Acceptor1 SelectChannelConnector@0.0.0.0:55383, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=19, name=qtp191968531-19 Acceptor0 SelectChannelConnector@0.0.0.0:55383, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=47, name=qtp1173231242-47 Acceptor1 SelectChannelConnector@0.0.0.0:21937, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=62, name=qtp1984873408-62 Acceptor2 SelectChannelConnector@0.0.0.0:54029, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=20, name=qtp191968531-20 Acceptor2 SelectChannelConnector@0.0.0.0:55383, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=21, name=qtp191968531-21 Acceptor3 SelectChannelConnector@0.0.0.0:55383, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=35, name=qtp1271263678-35 Acceptor3 SelectChannelConnector@0.0.0.0:31121, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=49, name=qtp1173231242-49 Acceptor3 SelectChannelConnector@0.0.0.0:21937, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=34, name=qtp1271263678-34 Acceptor2 SelectChannelConnector@0.0.0.0:31121, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=63, name=qtp1984873408-63 Acceptor3 SelectChannelConnector@0.0.0.0:54029, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 20 threads leaked from SUITE scope at org.apache.solr.update.SolrCmdDistributorTest: 
   1) Thread[id=61, name=qtp1984873408-61 Acceptor1 SelectChannelConnector@0.0.0.0:54029, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=48, name=qtp1173231242-48 Acceptor2 SelectChannelConnector@0.0.0.0:21937, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=33, name=qtp1271263678-33 Acceptor1 SelectChannelConnector@0.0.0.0:31121, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=60, name=qtp1984873408-60 Acceptor0 SelectChannelConnector@0.0.0.0:54029, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=74, name=qtp1189333092-74 Acceptor0 SelectChannelConnector@0.0.0.0:40308, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=76, name=qtp1189333092-76 Acceptor2 SelectChannelConnector@0.0.0.0:40308, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=77, name=qtp1189333092-77 Acceptor3 SelectChannelConnector@0.0.0.0:40308, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=75, name=qtp1189333092-75 Acceptor1 SelectChannelConnector@0.0.0.0:40308, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=46, name=qtp1173231242-46 Acceptor0 SelectChannelConnector@0.0.0.0:21937, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=32, name=qtp1271263678-32 Acceptor0 SelectChannelConnector@0.0.0.0:31121, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=15, name=qtp191968531-15 Acceptor1 SelectChannelConnector@0.0.0.0:55383, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=19, name=qtp191968531-19 Acceptor0 SelectChannelConnector@0.0.0.0:55383, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=47, name=qtp1173231242-47 Acceptor1 SelectChannelConnector@0.0.0.0:21937, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=62, name=qtp1984873408-62 Acceptor2 SelectChannelConnector@0.0.0.0:54029, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=20, name=qtp191968531-20 Acceptor2 SelectChannelConnector@0.0.0.0:55383, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=21, name=qtp191968531-21 Acceptor3 SelectChannelConnector@0.0.0.0:55383, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=35, name=qtp1271263678-35 Acceptor3 SelectChannelConnector@0.0.0.0:31121, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=49, name=qtp1173231242-49 Acceptor3 SelectChannelConnector@0.0.0.0:21937, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=34, name=qtp1271263678-34 Acceptor2 SelectChannelConnector@0.0.0.0:31121, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=63, name=qtp1984873408-63 Acceptor3 SelectChannelConnector@0.0.0.0:54029, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([43BDC8EDB8C9CCE8]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.update.SolrCmdDistributorTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=61, name=qtp1984873408-61 Acceptor1 SelectChannelConnector@0.0.0.0:54029, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=33, name=qtp1271263678-33 Acceptor1 SelectChannelConnector@0.0.0.0:31121, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=48, name=qtp1173231242-48 Acceptor2 SelectChannelConnector@0.0.0.0:21937, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=60, name=qtp1984873408-60 Acceptor0 SelectChannelConnector@0.0.0.0:54029, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=74, name=qtp1189333092-74 Acceptor0 SelectChannelConnector@0.0.0.0:40308, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=76, name=qtp1189333092-76 Acceptor2 SelectChannelConnector@0.0.0.0:40308, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=75, name=qtp1189333092-75 Acceptor1 SelectChannelConnector@0.0.0.0:40308, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=77, name=qtp1189333092-77 Acceptor3 SelectChannelConnector@0.0.0.0:40308, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=32, name=qtp1271263678-32 Acceptor0 SelectChannelConnector@0.0.0.0:31121, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=46, name=qtp1173231242-46 Acceptor0 SelectChannelConnector@0.0.0.0:21937, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=15, name=qtp191968531-15 Acceptor1 SelectChannelConnector@0.0.0.0:55383, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=19, name=qtp191968531-19 Acceptor0 SelectChannelConnector@0.0.0.0:55383, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=47, name=qtp1173231242-47 Acceptor1 SelectChannelConnector@0.0.0.0:21937, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=62, name=qtp1984873408-62 Acceptor2 SelectChannelConnector@0.0.0.0:54029, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=20, name=qtp191968531-20 Acceptor2 SelectChannelConnector@0.0.0.0:55383, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=21, name=qtp191968531-21 Acceptor3 SelectChannelConnector@0.0.0.0:55383, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=35, name=qtp1271263678-35 Acceptor3 SelectChannelConnector@0.0.0.0:31121, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=49, name=qtp1173231242-49 Acceptor3 SelectChannelConnector@0.0.0.0:21937, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=34, name=qtp1271263678-34 Acceptor2 SelectChannelConnector@0.0.0.0:31121, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=63, name=qtp1984873408-63 Acceptor3 SelectChannelConnector@0.0.0.0:54029, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=61, name=qtp1984873408-61 Acceptor1 SelectChannelConnector@0.0.0.0:54029, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=33, name=qtp1271263678-33 Acceptor1 SelectChannelConnector@0.0.0.0:31121, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=48, name=qtp1173231242-48 Acceptor2 SelectChannelConnector@0.0.0.0:21937, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=60, name=qtp1984873408-60 Acceptor0 SelectChannelConnector@0.0.0.0:54029, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=74, name=qtp1189333092-74 Acceptor0 SelectChannelConnector@0.0.0.0:40308, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=76, name=qtp1189333092-76 Acceptor2 SelectChannelConnector@0.0.0.0:40308, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=75, name=qtp1189333092-75 Acceptor1 SelectChannelConnector@0.0.0.0:40308, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=77, name=qtp1189333092-77 Acceptor3 SelectChannelConnector@0.0.0.0:40308, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=32, name=qtp1271263678-32 Acceptor0 SelectChannelConnector@0.0.0.0:31121, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=46, name=qtp1173231242-46 Acceptor0 SelectChannelConnector@0.0.0.0:21937, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=15, name=qtp191968531-15 Acceptor1 SelectChannelConnector@0.0.0.0:55383, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=19, name=qtp191968531-19 Acceptor0 SelectChannelConnector@0.0.0.0:55383, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=47, name=qtp1173231242-47 Acceptor1 SelectChannelConnector@0.0.0.0:21937, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=62, name=qtp1984873408-62 Acceptor2 SelectChannelConnector@0.0.0.0:54029, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=20, name=qtp191968531-20 Acceptor2 SelectChannelConnector@0.0.0.0:55383, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=21, name=qtp191968531-21 Acceptor3 SelectChannelConnector@0.0.0.0:55383, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=35, name=qtp1271263678-35 Acceptor3 SelectChannelConnector@0.0.0.0:31121, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=49, name=qtp1173231242-49 Acceptor3 SelectChannelConnector@0.0.0.0:21937, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=34, name=qtp1271263678-34 Acceptor2 SelectChannelConnector@0.0.0.0:31121, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=63, name=qtp1984873408-63 Acceptor3 SelectChannelConnector@0.0.0.0:54029, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([43BDC8EDB8C9CCE8]:0)


FAILED:  org.apache.solr.TestDistributedSearch.testDistribSearch

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([43BDC8EDB8C9CCE8]:0)




Build Log:
[...truncated 8298 lines...]
[junit4:junit4] Suite: org.apache.solr.update.SolrCmdDistributorTest
[junit4:junit4]   2> 276 T11 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> 914 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 1326 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:55383
[junit4:junit4]   2> 1529 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1530 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 1531 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2378 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 2383 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2384 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 2425 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 2430 T11 oasc.CoreContainer.<init> New CoreContainer 1019014919
[junit4:junit4]   2> 2432 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2432 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2532 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 2628 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 2630 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 2632 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 2633 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 2718 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 2898 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 2925 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 3015 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 4792 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 4829 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 4834 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5218 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5240 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5246 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5265 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5362 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/control/data/
[junit4:junit4]   2> 5363 T11 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 5431 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/control/data/index/
[junit4:junit4]   2> 5433 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1352114291911/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5553 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/control/data/index forceNew:false
[junit4:junit4]   2> 5976 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@2a16f5e lockFactory=org.apache.lucene.store.NativeFSLockFactory@296dfc2d; maxCacheMB=0.572265625 maxMergeSizeMB=0.55078125)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5977 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 7109 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 7135 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 7177 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 7184 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 7371 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 7481 T11 oass.SolrIndexSearcher.<init> Opening Searcher@834f574 main
[junit4:junit4]   2> 7482 T11 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 7605 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 7606 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 7738 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 7739 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 7740 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 7740 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 7741 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 7741 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 7742 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 7743 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 7743 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 7796 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 8429 T26 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@834f574 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 8432 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 8433 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 8434 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 8512 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 8538 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 8540 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:31121
[junit4:junit4]   2> 8542 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 8542 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 8543 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 8590 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 8591 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 8592 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 8592 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 8593 T11 oasc.CoreContainer.<init> New CoreContainer 156027432
[junit4:junit4]   2> 8594 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 8594 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 8630 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 8664 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 8665 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 8667 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 8668 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 8720 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 8779 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 8780 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 8832 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 9464 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 9481 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 9485 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 9508 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9514 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9519 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9522 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9525 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard0/data/
[junit4:junit4]   2> 9526 T11 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 9527 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard0/data/index/
[junit4:junit4]   2> 9528 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard0/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 9529 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard0/data/index forceNew:false
[junit4:junit4]   2> 9533 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@64712baa lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fc0dff4),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 9534 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 9536 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 9536 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 9537 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 9538 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 9553 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 9558 T11 oass.SolrIndexSearcher.<init> Opening Searcher@1373ef79 main
[junit4:junit4]   2> 9558 T11 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 9560 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 9560 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 9561 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 9562 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 9562 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 9563 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 9563 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 9564 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 9564 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 9565 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 9565 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 9566 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 9580 T40 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1373ef79 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 9579 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 9581 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 9581 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 9582 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9594 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 9597 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:21937
[junit4:junit4]   2> 9597 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 9598 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 9599 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 9642 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 9642 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 9643 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 9643 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 9644 T11 oasc.CoreContainer.<init> New CoreContainer 235102498
[junit4:junit4]   2> 9645 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 9645 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 9679 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 9700 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 9701 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 9702 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 9704 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 9750 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 9815 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 9816 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 9862 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 10428 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 10442 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 10446 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 10467 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10472 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10477 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10479 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10482 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard1/data/
[junit4:junit4]   2> 10483 T11 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 10484 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard1/data/index/
[junit4:junit4]   2> 10485 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard1/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 10486 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard1/data/index forceNew:false
[junit4:junit4]   2> 10489 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1416e4cd lockFactory=org.apache.lucene.store.NativeFSLockFactory@2f66531d),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 10490 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 10491 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 10492 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 10492 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 10493 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 10503 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 10513 T11 oass.SolrIndexSearcher.<init> Opening Searcher@6063eda7 main
[junit4:junit4]   2> 10513 T11 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 10514 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 10515 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 10516 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 10516 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 10517 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 10517 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 10518 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 10518 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 10519 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 10519 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 10520 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 10520 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 10534 T54 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6063eda7 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 10534 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 10535 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 10536 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 10537 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 10548 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 10551 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:54029
[junit4:junit4]   2> 10551 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 10552 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 10552 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 10584 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 10585 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 10586 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 10586 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 10587 T11 oasc.CoreContainer.<init> New CoreContainer 402434553
[junit4:junit4]   2> 10587 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 10588 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 10615 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 10634 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 10635 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 10637 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 10638 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 10675 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 10720 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 10721 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 10761 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 11356 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 11370 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 11373 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 11394 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 11399 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 11404 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 11406 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 11409 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard2/data/
[junit4:junit4]   2> 11409 T11 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 11410 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard2/data/index/
[junit4:junit4]   2> 11411 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard2/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 11412 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard2/data/index forceNew:false
[junit4:junit4]   2> 11416 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1eea7695 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2e367152),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 11416 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 11418 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 11418 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 11419 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 11420 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 11431 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 11437 T11 oass.SolrIndexSearcher.<init> Opening Searcher@5096a235 main
[junit4:junit4]   2> 11438 T11 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 11439 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 11440 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 11440 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 11441 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 11442 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 11442 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 11443 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 11443 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 11444 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 11444 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 11445 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 11445 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 11459 T68 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5096a235 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 11459 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 11460 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 11461 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 11461 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 11473 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 11476 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:40308
[junit4:junit4]   2> 11476 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 11477 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 11477 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 11511 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 11512 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 11512 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 11513 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 11513 T11 oasc.CoreContainer.<init> New CoreContainer 82190788
[junit4:junit4]   2> 11514 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 11515 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 11544 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 11562 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 11563 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 11564 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 11565 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 11612 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 11661 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 11662 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 11696 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 12292 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 12309 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 12313 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 12333 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 12338 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 12343 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 12345 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 12347 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard3/data/
[junit4:junit4]   2> 12347 T11 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 12349 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard3/data/index/
[junit4:junit4]   2> 12350 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard3/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 12350 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard3/data/index forceNew:false
[junit4:junit4]   2> 12354 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@9a99335 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6390f540),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 12354 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 12356 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 12356 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 12357 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 12357 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 12367 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 12372 T11 oass.SolrIndexSearcher.<init> Opening Searcher@753d011 main
[junit4:junit4]   2> 12373 T11 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 12374 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 12375 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 12375 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 12376 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 12376 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 12377 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 12377 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 12378 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 12378 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 12379 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 12379 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 12380 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 12395 T82 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@753d011 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 12395 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 12396 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 12397 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 12397 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C0 name=collection1 org.apache.solr.core.SolrCore@6e21b5ab
[junit4:junit4]   2> 14154 T24 C0 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@2a16f5e lockFactory=org.apache.lucene.store.NativeFSLockFactory@296dfc2d; maxCacheMB=0.572265625 maxMergeSizeMB=0.55078125)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 14156 T24 C0 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 14203 T24 C0 oass.SolrIndexSearcher.<init> Opening Searcher@31394b77 realtime
[junit4:junit4]   2> 14204 T24 C0 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 14211 T24 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1417794610009210880)} 0 620
[junit4:junit4]   2> ASYNC  NEW_CORE C1 name=collection1 org.apache.solr.core.SolrCore@27e1d8a2
[junit4:junit4]   2> 14405 T38 C1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@64712baa lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fc0dff4),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 14406 T38 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 14408 T38 C1 oass.SolrIndexSearcher.<init> Opening Searcher@7c004572 realtime
[junit4:junit4]   2> 14408 T38 C1 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 14409 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1417794610562859008)} 0 7
[junit4:junit4]   2> ASYNC  NEW_CORE C2 name=collection1 org.apache.solr.core.SolrCore@18c18838
[junit4:junit4]   2> 14422 T53 C2 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1416e4cd lockFactory=org.apache.lucene.store.NativeFSLockFactory@2f66531d),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 14423 T53 C2 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 14425 T53 C2 oass.SolrIndexSearcher.<init> Opening Searcher@2ad2ee23 realtime
[junit4:junit4]   2> 14426 T53 C2 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 14427 T53 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1417794610578587648)} 0 9
[junit4:junit4]   2> ASYNC  NEW_CORE C3 name=collection1 org.apache.solr.core.SolrCore@701eec73
[junit4:junit4]   2> 14446 T65 C3 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1eea7695 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2e367152),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 14447 T65 C3 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 14449 T65 C3 oass.SolrIndexSearcher.<init> Opening Searcher@5141185c realtime
[junit4:junit4]   2> 14450 T65 C3 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 14450 T65 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1417794610600607744)} 0 11
[junit4:junit4]   2> ASYNC  NEW_CORE C4 name=collection1 org.apache.solr.core.SolrCore@429302e2
[junit4:junit4]   2> 14461 T80 C4 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@9a99335 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6390f540),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 14461 T80 C4 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 14463 T80 C4 oass.SolrIndexSearcher.<init> Opening Searcher@24ed587c realtime
[junit4:junit4]   2> 14463 T80 C4 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 14464 T80 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1417794610620530688)} 0 6
[junit4:junit4]   2> 14486 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16
[junit4:junit4]   2> 14961 T22 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1 (1417794610794594304)]} 0 348
[junit4:junit4]   2> 14969 T11 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:55383/solr/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4]   2> 14993 T22 C0 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15119 T22 C0 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@2a16f5e lockFactory=org.apache.lucene.store.NativeFSLockFactory@296dfc2d; maxCacheMB=0.572265625 maxMergeSizeMB=0.55078125)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@2a16f5e lockFactory=org.apache.lucene.store.NativeFSLockFactory@296dfc2d; maxCacheMB=0.572265625 maxMergeSizeMB=0.55078125)),segFN=segments_2,generation=2,filenames=[_0_Lucene40_0.tim, _0.fnm, _0_Lucene40_0.frq, _0_nrm.cfe, segments_2, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene40_0.prx, _0.fdt, _0_Lucene40_0.tip]
[junit4:junit4]   2> 15119 T22 C0 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 15251 T22 C0 oass.SolrIndexSearcher.<init> Opening Searcher@53670068 main
[junit4:junit4]   2> 15252 T22 C0 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 15256 T22 C0 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15269 T26 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@53670068 main{StandardDirectoryReader(segments_2:3:nrt _0(4.1):C1)}
[junit4:junit4]   2> 15270 T22 C0 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&commit_end_point=true&expungeDeletes=false&wt=javabin&softCommit=false&version=2} {commit=} 0 277
[junit4:junit4]   2> 15545 T23 C0 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=1 status=0 QTime=243 
[junit4:junit4]   2> 15602 T22 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&commit_end_point=true&version=2} {add=[2 (1417794611815907328), 3 (1417794611819053056)]} 0 5
[junit4:junit4]   2> 15603 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&commit_end_point=true&version=2} {add=[2 (1417794611814858752), 3 (1417794611816955904), 4 (1417794611819053056)]} 0 6
[junit4:junit4]   2> 15604 T11 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:55383/solr/, StdNode: http://127.0.0.1:31121/solr/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4]   2> 15608 T24 C0 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15608 T37 C1 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15613 T37 C1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@64712baa lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fc0dff4),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@64712baa lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fc0dff4),segFN=segments_2,generation=2,filenames=[_0_Lucene40_0.tim, _0.fnm, _0_Lucene40_0.frq, _0_nrm.cfe, segments_2, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene40_0.prx, _0.fdt, _0_Lucene40_0.tip]
[junit4:junit4]   2> 15614 T37 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 15616 T37 C1 oass.SolrIndexSearcher.<init> Opening Searcher@6516b1d9 main
[junit4:junit4]   2> 15616 T37 C1 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 15618 T40 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6516b1d9 main{StandardDirectoryReader(segments_2:3:nrt _0(4.1):C3)}
[junit4:junit4]   2> 15618 T37 C1 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15618 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&commit_end_point=true&expungeDeletes=false&wt=javabin&softCommit=false&version=2} {commit=} 0 10
[junit4:junit4]   2> 15658 T24 C0 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@2a16f5e lockFactory=org.apache.lucene.store.NativeFSLockFactory@296dfc2d; maxCacheMB=0.572265625 maxMergeSizeMB=0.55078125)),segFN=segments_2,generation=2,filenames=[_0_Lucene40_0.tim, _0.fnm, _0_Lucene40_0.frq, _0_nrm.cfe, segments_2, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene40_0.prx, _0.fdt, _0_Lucene40_0.tip]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@2a16f5e lockFactory=org.apache.lucene.store.NativeFSLockFactory@296dfc2d; maxCacheMB=0.572265625 maxMergeSizeMB=0.55078125)),segFN=segments_3,generation=3,filenames=[_1_Lucene40_0.tim, _1_nrm.cfs, _0_Lucene40_0.tim, _0.fnm, _1_Lucene40_0.frq, _0_nrm.cfs, _0_Lucene40_0.prx, _1.fnm, _1_Lucene40_0.prx, _1_Lucene40_0.tip, _1_nrm.cfe, _0_Lucene40_0.tip, _1.fdx, _0_Lucene40_0.frq, _1.fdt, _1.si, _0_nrm.cfe, _0.fdx, _0.si, segments_3, _0.fdt]
[junit4:junit4]   2> 15659 T24 C0 oasc.SolrDeletionPolicy.updateCommits newest commit = 3
[junit4:junit4]   2> 15668 T24 C0 oass.SolrIndexSearcher.<init> Opening Searcher@586d9632 main
[junit4:junit4]   2> 15668 T24 C0 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 15669 T24 C0 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15669 T26 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@586d9632 main{StandardDirectoryReader(segments_3:5:nrt _0(4.1):C1 _1(4.1):C2)}
[junit4:junit4]   2> 15670 T24 C0 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&commit_end_point=true&expungeDeletes=false&wt=javabin&softCommit=false&version=2} {commit=} 0 62
[junit4:junit4]   2> 15674 T23 C0 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=3 status=0 QTime=1 
[junit4:junit4]   2> 15683 T38 C1 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=3 status=0 QTime=1 
[junit4:junit4]   2> 15691 T22 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&commit_end_point=true&version=2} {delete=[2 (-1417794611912376320)]} 0 1
[junit4:junit4]   2> 15691 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&commit_end_point=true&version=2} {delete=[2 (-1417794611912376320)]} 0 1
[junit4:junit4]   2> 15693 T11 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:55383/solr/, StdNode: http://127.0.0.1:31121/solr/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4]   2> 15696 T37 C1 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15696 T24 C0 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15710 T37 C1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@64712baa lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fc0dff4),segFN=segments_2,generation=2,filenames=[_0_Lucene40_0.tim, _0.fnm, _0_Lucene40_0.frq, _0_nrm.cfe, segments_2, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene40_0.prx, _0.fdt, _0_Lucene40_0.tip]
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@64712baa lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fc0dff4),segFN=segments_3,generation=3,filenames=[_0_Lucene40_0.tim, _0.fnm, _0_Lucene40_0.frq, _0_nrm.cfe, _0_1.del, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene40_0.prx, _0.fdt, segments_3, _0_Lucene40_0.tip]
[junit4:junit4]   2> 15711 T37 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 3
[junit4:junit4]   2> 15711 T24 C0 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@2a16f5e lockFactory=org.apache.lucene.store.NativeFSLockFactory@296dfc2d; maxCacheMB=0.572265625 maxMergeSizeMB=0.55078125)),segFN=segments_3,generation=3,filenames=[_1_Lucene40_0.tim, _1_nrm.cfs, _0_Lucene40_0.tim, _0.fnm, _1_Lucene40_0.frq, _0_nrm.cfs, _0_Lucene40_0.prx, _1.fnm, _1_Lucene40_0.prx, _1_Lucene40_0.tip, _1_nrm.cfe, _0_Lucene40_0.tip, _1.fdx, _0_Lucene40_0.frq, _1.fdt, _1.si, _0_nrm.cfe, _0.fdx, _0.si, segments_3, _0.fdt]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@2a16f5e lockFactory=org.apache.lucene.store.NativeFSLockFactory@296dfc2d; maxCacheMB=0.572265625 maxMergeSizeMB=0.55078125)),segFN=segments_4,generation=4,filenames=[_1_Lucene40_0.tim, _1_nrm.cfs, _0_Lucene40_0.tim, _0.fnm, _1_Lucene40_0.frq, _0_nrm.cfs, _1_1.del, _0_Lucene40_0.prx, _1.fnm, _1_Lucene40_0.prx, _1_Lucene40_0.tip, _1_nrm.cfe, _0_Lucene40_0.tip, _1.fdx, _0_Lucene40_0.frq, _1.fdt, _1.si, _0_nrm.cfe, _0.fdx, _0.si, _0.fdt, segments_4]
[junit4:junit4]   2> 15712 T37 C1 oass.SolrIndexSearcher.<init> Opening Searcher@14b58fc0 main
[junit4:junit4]   2> 15712 T24 C0 oasc.SolrDeletionPolicy.updateCommits newest commit = 4
[junit4:junit4]   2> 15712 T37 C1 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 15713 T40 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@14b58fc0 main{StandardDirectoryReader(segments_3:4:nrt _0(4.1):C3/1)}
[junit4:junit4]   2> 15713 T37 C1 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15714 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&commit_end_point=true&expungeDeletes=false&wt=javabin&softCommit=false&version=2} {commit=} 0 18
[junit4:junit4]   2> 15716 T24 C0 oass.SolrIndexSearcher.<init> Opening Searcher@15ec1723 main
[junit4:junit4]   2> 15717 T24 C0 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 15718 T26 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@15ec1723 main{StandardDirectoryReader(segments_4:6:nrt _0(4.1):C1 _1(4.1):C2/1)}
[junit4:junit4]   2> 15718 T24 C0 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15719 T24 C0 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&commit_end_point=true&expungeDeletes=false&wt=javabin&softCommit=false&version=2} {commit=} 0 23
[junit4:junit4]   2> 15723 T23 C0 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=2 status=0 QTime=1 
[junit4:junit4]   2> 15728 T38 C1 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=2 status=0 QTime=1 
[junit4:junit4]   2> 15732 T39 C1 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15974 T39 C1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@64712baa lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fc0dff4),segFN=segments_3,generation=3,filenames=[_0_Lucene40_0.tim, _0.fnm, _0_Lucene40_0.frq, _0_nrm.cfe, _0_1.del, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene40_0.prx, _0.fdt, segments_3, _0_Lucene40_0.tip]
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@64712baa lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fc0dff4),segFN=segments_4,generation=4,filenames=[_1_Lucene40_0.tim, _1.fdx, _1_nrm.cfs, _1.fdt, _1.si, _1_Lucene40_0.frq, _1.fnm, _1_Lucene40_0.prx, _1_Lucene40_0.tip, _1_nrm.cfe, segments_4]
[junit4:junit4]   2> 15975 T39 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 4
[junit4:junit4]   2> 15979 T39 C1 oass.SolrIndexSearcher.<init> Opening Searcher@75a3a7f7 main
[junit4:junit4]   2> 15979 T39 C1 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 15980 T39 C1 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15981 T40 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@75a3a7f7 main{StandardDirectoryReader(segments_4:6:nrt _1(4.1):C2)}
[junit4:junit4]   2> 15982 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={optimize=true&maxSegments=1&waitSearcher=true&wt=javabin&version=2} {optimize=} 0 250
[junit4:junit4]   2> 15987 T52 C2 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15989 T52 C2 oass.SolrIndexSearcher.<init> Opening Searcher@4a0a4516 main
[junit4:junit4]   2> 15989 T52 C2 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 15990 T54 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4a0a4516 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4]   2> 15990 T52 C2 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15991 T52 C2 UPDATE [collection1] webapp=/solr path=/update params={optimize=true&maxSegments=1&waitSearcher=true&wt=javabin&version=2} {optimize=} 0 4
[junit4:junit4]   2> 15995 T67 C3 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15996 T67 C3 oass.SolrIndexSearcher.<init> Opening Searcher@4038e358 main
[junit4:junit4]   2> 15997 T67 C3 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 15998 T68 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4038e358 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4]   2> 15998 T67 C3 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15998 T67 C3 UPDATE [collection1] webapp=/solr path=/update params={optimize=true&maxSegments=1&waitSearcher=true&wt=javabin&version=2} {optimize=} 0 3
[junit4:junit4]   2> 16002 T78 C4 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 16004 T78 C4 oass.SolrIndexSearcher.<init> Opening Searcher@401bb340 main
[junit4:junit4]   2> 16004 T78 C4 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 16005 T82 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@401bb340 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4]   2> 16005 T78 C4 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 16006 T78 C4 UPDATE [collection1] webapp=/solr path=/update params={optimize=true&maxSegments=1&waitSearcher=true&wt=javabin&version=2} {optimize=} 0 4
[junit4:junit4]   2> 16058 T81 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[5 (1417794612268892160), 7 (1417794612276232192), 9 (1417794612278329344), 10 (1417794612281475073), 11 (1417794612283572225), 12 (1417794612285669376), 14 (1417794612287766528), 17 (1417794612289863680), 18 (1417794612293009409), 20 (1417794612295106560)]} 0 27
[junit4:junit4]   2> 16072 T78 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[21 (1417794612273086464), 22 (1417794612276232193), 23 (1417794612281475072), 24 (1417794612283572224), 25 (1417794612284620800), 26 (1417794612285669377), 30 (1417794612286717952), 31 (1417794612287766529), 32 (1417794612289863681), 34 (1417794612304543744)]} 0 38
[junit4:junit4]   2> 16084 T65 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[5 (1417794612270989312), 7 (1417794612276232193), 9 (1417794612281475072), 11 (1417794612291960833), 12 (1417794612297203712), 16 (1417794612300349440), 19 (1417794612303495168), 21 (1417794612307689473), 23 (1417794612312932353), 25 (1417794612317126656)]} 0 53
[junit4:junit4]   2> 16089 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[49 (1417794612289863680), 50 (1417794612295106561), 51 (1417794612296155137), 53 (1417794612298252288), 54 (1417794612300349440), 56 (1417794612301398017), 58 (1417794612302446593), 59 (1417794612307689473), 60 (1417794612313980928), 61 (1417794612328660993)]} 0 39
[junit4:junit4]   2> 16090 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[35 (1417794612274135040), 36 (1417794612291960832), 37 (1417794612293009408), 38 (1417794612296155136), 40 (1417794612299300864), 41 (1417794612301398016), 42 (1417794612307689472), 44 (1417794612312932352), 46 (1417794612315029504), 48 (1417794612317126656)]} 0 55
[junit4:junit4]   2> 16091 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[6 (1417794612272037888), 13 (1417794612279377920), 15 (1417794612291960833), 22 (1417794612295106560), 27 (1417794612297203712), 28 (1417794612300349441), 29 (1417794612302446592), 30 (1417794612309786624), 31 (1417794612328660992), 32 (1417794612330758144)]} 0 59
[junit4:junit4]   2> 16099 T52 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[5 (1417794612269940736), 7 (1417794612275183616), 10 (1417794612288815105), 12 (1417794612293009408), 13 (1417794612299300864), 17 (1417794612302446592), 19 (1417794612307689472), 20 (1417794612311883776), 22 (1417794612332855298), 23 (1417794612338098177)]} 0 69
[junit4:junit4]   2> 16101 T66 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[27 (1417794612276232192), 28 (1417794612291960832), 29 (1417794612293009408), 33 (1417794612298252288), 34 (1417794612305592320), 37 (1417794612309786624), 39 (1417794612315029505), 42 (1417794612318175232), 43 (1417794612326563840), 44 (1417794612328660992)]} 0 65
[junit4:junit4]   2> 16116 T67 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[47 (1417794612289863680), 50 (1417794612312932352), 52 (1417794612323418112), 53 (1417794612327612416), 54 (1417794612329709568), 58 (1417794612343341056), 61 (1417794612347535360), 63 (1417794612350681089), 64 (1417794612353826816), 65 (1417794612355923968)]} 0 66
[junit4:junit4]   2> 16116 T106 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[64 (1417794612310835200), 66 (1417794612327612416), 68 (1417794612333903872), 74 (1417794612342292480), 79 (1417794612345438208), 81 (1417794612346486784), 82 (1417794612347535361), 85 (1417794612350681088), 86 (1417794612352778240), 88 (1417794612355923968)]} 0 46
[junit4:junit4]   2> 16126 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[183 (1417794612347535360), 184 (1417794612349632512), 185 (1417794612351729664), 189 (1417794612354875392), 190 (1417794612358021120), 191 (1417794612360118272), 192 (1417794612362215424), 193 (1417794612363264000), 194 (1417794612365361152), 195 (1417794612367458304)]} 0 22
[junit4:junit4]   2> 16150 T100 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[64 (1417794612298252288), 67 (1417794612308738048), 68 (1417794612328660992), 69 (1417794612332855296), 72 (1417794612334952450), 73 (1417794612341243904), 74 (1417794612344389632), 76 (1417794612345438208), 78 (1417794612347535360), 79 (1417794612389478400)]} 0 93
[junit4:junit4]   2> 16154 T65 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[108 (1417794612339146752), 113 (1417794612346486784), 129 (1417794612348583936), 131 (1417794612350681088), 133 (1417794612351729664), 140 (1417794612354875392), 142 (1417794612358021120), 143 (1417794612360118272), 144 (1417794612362215424), 147 (1417794612394721280)]} 0 58
[junit4:junit4]   2> 16159 T53 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[27 (1417794612273086464), 29 (1417794612288815104), 31 (1417794612291960832), 33 (1417794612297203712), 36 (1417794612300349440), 37 (1417794612303495168), 39 (1417794612309786624), 41 (1417794612323418113), 42 (1417794612332855296), 46 (1417794612336001024)]} 0 125
[junit4:junit4]   2> 16161 T80 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[35 (1417794612277280768), 41 (1417794612288815104), 43 (1417794612293009408), 45 (1417794612306640896), 47 (1417794612322369537), 52 (1417794612329709568), 55 (1417794612330758145), 56 (1417794612334952448), 60 (1417794612392624129), 61 (1417794612397867008)]} 0 123
[junit4:junit4]   2> 16172 T102 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[66 (1417794612307689472), 67 (1417794612315029504), 68 (1417794612332855296), 69 (1417794612345438208), 70 (1417794612349632512), 72 (1417794612352778240), 74 (1417794612395769856), 76 (1417794612399964161), 77 (1417794612410449920), 78 (1417794612413595648)]} 0 106
[junit4:junit4]   2> 16173 T112 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[101 (1417794612323418112), 102 (1417794612330758144), 103 (1417794612333903873), 108 (1417794612387381248), 109 (1417794612407304192), 110 (1417794612409401345), 111 (1417794612412547072), 112 (1417794612413595648), 114 (1417794612415692800), 115 (1417794612416741377)]} 0 91
[junit4:junit4]   2> 16176 T51 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[50 (1417794612289863680), 52 (1417794612308738048), 53 (1417794612315029504), 56 (1417794612317126656), 57 (1417794612323418114), 59 (1417794612328660992), 60 (1417794612338098176), 64 (1417794612341243904), 65 (1417794612416741378), 66 (1417794612418838528)]} 0 126
[junit4:junit4]   2> 16186 T108 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[90 (1417794612316078080), 91 (1417794612330758145), 92 (1417794612332855296), 93 (1417794612387381248), 95 (1417794612391575552), 96 (1417794612396818434), 97 (1417794612401012736), 102 (1417794612409401344), 103 (1417794612413595648), 107 (1417794612426178562)]} 0 111
[junit4:junit4]   2> 16189 T105 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[85 (1417794612313980928), 86 (1417794612325515264), 92 (1417794612328660993), 94 (1417794612394721281), 96 (1417794612397867008), 101 (1417794612399964160), 102 (1417794612406255616), 103 (1417794612409401344), 106 (1417794612426178560), 107 (1417794612431421440)]} 0 117
[junit4:junit4]   2> 16190 T81 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[81 (1417794612310835200), 84 (1417794612315029504), 86 (1417794612317126656), 87 (1417794612322369536), 89 (1417794612324466688), 91 (1417794612329709569), 92 (1417794612334952449), 93 (1417794612392624128), 95 (1417794612397867009), 100 (1417794612423032833)]} 0 120
[junit4:junit4]   2> 16198 T114 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[108 (1417794612331806720), 111 (1417794612393672704), 112 (1417794612398915584), 115 (1417794612407304192), 116 (1417794612410449920), 119 (1417794612423032833), 122 (1417794612426178563), 123 (1417794612428275714), 124 (1417794612437712897), 125 (1417794612439810048)]} 0 114
[junit4:junit4]   2> 16200 T125 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[133 (1417794612342292480), 135 (1417794612391575553), 138 (1417794612395769856), 140 (1417794612399964160), 141 (1417794612408352768), 146 (1417794612424081408), 147 (1417794612427227137), 148 (1417794612431421440), 149 (1417794612434567169), 153 (1417794612438761473)]} 0 101
[junit4:junit4]   2> 16202 T103 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[85 (1417794612312932353), 92 (1417794612316078080), 93 (1417794612323418112), 94 (1417794612326563840), 95 (1417794612334952448), 98 (1417794612339146753), 100 (1417794612404158464), 102 (1417794612416741379), 103 (1417794612419887104), 106 (1417794612421984256)]} 0 130
[junit4:junit4]   2> 16204 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[131 (1417794612339146752), 134 (1417794612389478400), 135 (1417794612392624129), 136 (1417794612396818433), 138 (1417794612398915585), 139 (1417794612405207040), 142 (1417794612424081408), 143 (1417794612428275713), 144 (1417794612430372864), 145 (1417794612448198656)]} 0 108
[junit4:junit4]   2> 16209 T122 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[138 (1417794612339146752), 144 (1417794612416741376), 145 (1417794612424081408), 146 (1417794612427227136), 147 (1417794612429324288), 148 (1417794612431421441), 150 (1417794612433518593), 153 (1417794612435615745), 156 (1417794612452392960), 157 (1417794612454490112)]} 0 113
[junit4:junit4]   2> 16216 T67 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[192 (1417794612399964162), 194 (1417794612405207041), 197 (1417794612412547072), 203 (1417794612415692800), 204 (1417794612421984256), 207 (1417794612427227137), 208 (1417794612431421441), 211 (1417794612435615744), 212 (1417794612438761473), 213 (1417794612455538691)]} 0 61
[junit4:junit4]   2> 16231 T128 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[154 (1417794612391575552), 155 (1417794612409401344), 162 (1417794612426178560), 163 (1417794612429324288), 164 (1417794612432470018), 165 (1417794612437712896), 166 (1417794612440858624), 167 (1417794612449247234), 174 (1417794612453441536), 175 (1417794612460781569)]} 0 84
[junit4:junit4]   2> 16238 T101 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[68 (1417794612298252288), 69 (1417794612312932352), 71 (1417794612333903872), 72 (1417794612341243905), 73 (1417794612416741377), 74 (1417794612428275712), 77 (1417794612431421440), 78 (1417794612433518592), 80 (1417794612435615744), 83 (1417794612453441536)]} 0 181
[junit4:junit4]   2> 16239 T126 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[165 (1417794612396818432), 167 (1417794612416741376), 170 (1417794612419887105), 171 (1417794612421984256), 172 (1417794612433518592), 173 (1417794612437712896), 174 (1417794612448198657), 175 (1417794612452392961), 178 (1417794612462878720), 182 (1417794612480704512)]} 0 88
[junit4:junit4]   2> 16241 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[274 (1417794612412547072), 275 (1417794612415692800), 276 (1417794612419887104), 277 (1417794612423032832), 278 (1417794612435615744), 280 (1417794612438761473), 281 (1417794612453441536), 283 (1417794612457635841), 284 (1417794612479655936), 285 (1417794612486995968)]} 0 73
[junit4:junit4]   2> 16246 T146 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[268 (1417794612421984257), 269 (1417794612427227136), 270 (1417794612432470016), 271 (1417794612434567169), 272 (1417794612437712896), 274 (1417794612439810048), 275 (1417794612446101504), 276 (1417794612452392960), 277 (1417794612454490113), 279 (1417794612490141696)]} 0 70
[junit4:junit4]   2> 16254 T100 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[218 (1417794612405207042), 219 (1417794612412547073), 222 (1417794612414644224), 223 (1417794612416741376), 225 (1417794612418838528), 228 (1417794612420935680), 229 (1417794612423032832), 230 (1417794612445052928), 232 (1417794612460781568), 233 (1417794612497481728)]} 0 95
[junit4:junit4]   2> 16264 T136 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[235 (1417794612432470016), 236 (1417794612449247232), 237 (1417794612453441538), 239 (1417794612455538690), 240 (1417794612461830144), 241 (1417794612480704513), 242 (1417794612486995968), 244 (1417794612503773184), 246 (1417794612507967488), 247 (1417794612511113217)]} 0 78
[junit4:junit4]   2> 16265 T66 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[174 (1417794612390526976), 176 (1417794612404158464), 178 (1417794612425129984), 184 (1417794612444004352), 185 (1417794612452392961), 186 (1417794612458684416), 187 (1417794612495384576), 188 (1417794612499578880), 189 (1417794612507967491), 191 (1417794612511113216)]} 0 119
[junit4:junit4]   2> 16267 T129 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[149 (1417794612344389632), 150 (1417794612392624128), 152 (1417794612416741376), 153 (1417794612419887105), 154 (1417794612444004353), 156 (1417794612447150080), 160 (1417794612450295809), 164 (1417794612455538689), 169 (1417794612481753088), 171 (1417794612514258944)]} 0 166
[junit4:junit4]   2> 16267 T80 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[271 (1417794612432470017), 274 (1417794612436664320), 276 (1417794612439810049), 277 (1417794612447150081), 278 (1417794612451344384), 282 (1417794612455538689), 283 (1417794612457635840), 291 (1417794612477558784), 292 (1417794612509016065), 293 (1417794612515307520)]} 0 81
[junit4:junit4]   2> 16267 T51 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[220 (1417794612433518594), 222 (1417794612439810048), 227 (1417794612455538689), 228 (1417794612457635841), 233 (1417794612495384576), 234 (1417794612502724608), 235 (1417794612504821761), 239 (1417794612507967488), 242 (1417794612510064640), 244 (1417794612512161792)]} 0 80
[junit4:junit4]   2> 16292 T117 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[107 (1417794612332855297), 111 (1417794612387381248), 112 (1417794612450295808), 114 (1417794612453441537), 118 (1417794612457635840), 119 (1417794612478607362), 124 (1417794612483850241), 128 (1417794612489093121), 133 (1417794612503773184), 137 (1417794612506918913)]} 0 201
[junit4:junit4]   2> 16293 T106 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[229 (1417794612410449921), 231 (1417794612426178561), 232 (1417794612444004354), 235 (1417794612448198658), 238 (1417794612452392960), 240 (1417794612456587264), 242 (1417794612486995969), 246 (1417794612495384577), 247 (1417794612498530304), 250 (1417794612513210368)]} 0 128
[junit4:junit4]   2> 16288 T103 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[334 (1417794612488044544), 336 (1417794612506918912), 338 (1417794612522647555), 339 (1417794612524744706), 340 (1417794612526841856), 341 (1417794612528939011), 344 (1417794612531036162), 346 (1417794612534181889), 347 (1417794612536279040), 350 (1417794612537327616)]} 0 50
[junit4:junit4]   2> 16286 T147 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[248 (1417794612451344385), 250 (1417794612455538688), 254 (1417794612478607361), 255 (1417794612483850240), 259 (1417794612505870337), 260 (1417794612522647552), 261 (1417794612525793283), 262 (1417794612528939009), 263 (1417794612532084737), 265 (1417794612535230464)]} 0 88
[junit4:junit4]   2> 16295 T108 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[252 (1417794612434567168), 253 (1417794612436664320), 254 (1417794612438761472), 255 (1417794612440858624), 258 (1417794612449247232), 263 (1417794612453441538), 265 (1417794612458684416), 266 (1417794612479655937), 269 (1417794612483850240), 270 (1417794612489093121)]} 0 108
[junit4:junit4]   2> 16285 T151 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[309 (1417794612478607360), 311 (1417794612486995968), 312 (1417794612493287424), 314 (1417794612497481728), 318 (1417794612505870338), 323 (1417794612522647553), 324 (1417794612525793280), 326 (1417794612528939008), 327 (1417794612532084738), 333 (1417794612534181888)]} 0 56
[junit4:junit4]   2> 16285 T145 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[266 (1417794612420935681), 267 (1417794612460781568), 268 (1417794612481753089), 269 (1417794612489093120), 270 (1417794612497481729), 274 (1417794612499578882), 276 (1417794612524744705), 278 (1417794612527890434), 282 (1417794612531036160), 285 (1417794612534181891)]} 0 110
[junit4:junit4]   2> 16283 T122 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[351 (1417794612496433152), 354 (1417794612505870336), 359 (1417794612515307521), 361 (1417794612519501824), 362 (1417794612522647554), 363 (1417794612524744704), 365 (1417794612525793282), 368 (1417794612527890433), 371 (1417794612529987584), 372 (1417794612532084739)]} 0 36
[junit4:junit4]   2> 16276 T52 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[178 (1417794612395769856), 180 (1417794612451344384), 181 (1417794612455538690), 182 (1417794612463927297), 183 (1417794612480704512), 184 (1417794612482801664), 188 (1417794612489093122), 189 (1417794612499578881), 190 (1417794612521598976), 192 (1417794612523696129)]} 0 126
[junit4:junit4]   2> 16303 T120 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[198 (1417794612397867008), 201 (1417794612418838528), 206 (1417794612426178560), 208 (1417794612428275712), 209 (1417794612444004353), 210 (1417794612453441537), 213 (1417794612457635840), 224 (1417794612478607360), 227 (1417794612481753088), 228 (1417794612495384576)]} 0 150
[junit4:junit4]   2> 16275 T112 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[252 (1417794612425129984), 253 (1417794612427227136), 255 (1417794612428275712), 258 (1417794612449247233), 260 (1417794612480704512), 261 (1417794612510064641), 262 (1417794612513210368), 266 (1417794612519501824), 268 (1417794612521598976), 269 (1417794612523696129)]} 0 96
[junit4:junit4]   2> 16273 T133 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[197 (1417794612405207041), 198 (1417794612435615744), 202 (1417794612439810048), 203 (1417794612447150080), 204 (1417794612451344385), 205 (1417794612455538688), 206 (1417794612458684416), 208 (1417794612493287425), 214 (1417794612517404672), 216 (1417794612519501826)]} 0 114
[junit4:junit4]   2> 16270 T53 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[194 (1417794612409401344), 199 (1417794612420935680), 200 (1417794612448198656), 201 (1417794612457635842), 203 (1417794612477558784), 204 (1417794612481753088), 208 (1417794612494336000), 212 (1417794612499578880), 213 (1417794612515307520), 215 (1417794612517404672)]} 0 106
[junit4:junit4]   2> 16304 T81 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[294 (1417794612462878720), 295 (1417794612494336000), 296 (1417794612499578880), 297 (1417794612506918914), 300 (1417794612509016066), 302 (1417794612545716224), 303 (1417794612548861952), 304 (1417794612550959105), 305 (1417794612553056256), 308 (1417794612555153408)]} 0 90
[junit4:junit4]   2> 16294 T101 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[287 (1417794612495384577), 290 (1417794612520550401), 291 (1417794612525793281), 298 (1417794612528939010), 299 (1417794612531036161), 301 (1417794612534181890), 304 (1417794612536279041), 305 (1417794612539424768), 306 (1417794612541521920), 307 (1417794612543619072)]} 0 48
[junit4:junit4]   2> 16293 T100 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[371 (1417794612523696128), 372 (1417794612525793280), 373 (1417794612527890432), 375 (1417794612529987584), 377 (1417794612532084736), 379 (1417794612534181888), 380 (1417794612536279040), 381 (1417794612537327616), 382 (1417794612539424768), 383 (1417794612540473344)]} 0 21
[junit4:junit4]   2> 16293 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[147 (1417794612339146753), 148 (1417794612392624128), 149 (1417794612396818435), 152 (1417794612399964160), 155 (1417794612406255616), 156 (1417794612444004352), 157 (1417794612460781568), 161 (1417794612490141696), 162 (1417794612505870336), 164 (1417794612516356096)]} 0 197
[junit4:junit4]   2> 16310 T102 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[252 (1417794612419887104), 253 (1417794612425129985), 255 (1417794612429324288), 257 (1417794612433518592), 260 (1417794612438761472), 261 (1417794612448198656), 262 (1417794612455538688), 264 (1417794612457635840), 265 (1417794612507967492), 266 (1417794612550959105)]} 0 136
[junit4:junit4]   2> 16311 T78 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[116 (1417794612333903872), 118 (1417794612339146752), 120 (1417794612387381249), 124 (1417794612392624130), 125 (1417794612408352769), 128 (1417794612421984256), 129 (1417794612446101504), 130 (1417794612479655936), 131 (1417794612511113218), 132 (1417794612549910528)]} 0 219
[junit4:junit4]   2> 16292 T132 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[179 (1417794612405207040), 182 (1417794612434567168), 183 (1417794612438761472), 184 (1417794612440858625), 185 (1417794612448198656), 187 (1417794612453441537), 191 (1417794612456587264), 194 (1417794612490141696), 195 (1417794612509016064), 196 (1417794612512161793)]} 0 133
[junit4:junit4]   2> 16291 T127 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[158 (1417794612390526976), 161 (1417794612463927296), 165 (1417794612504821760), 166 (1417794612520550400), 167 (1417794612523696128), 171 (1417794612527890432), 172 (1417794612532084736), 173 (1417794612535230465), 174 (1417794612537327617), 177 (1417794612540473344)]} 0 145
[junit4:junit4]   2> 16312 T65 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[214 (1417794612407304192), 216 (1417794612426178561), 218 (1417794612430372864), 224 (1417794612434567168), 225 (1417794612454490112), 226 (1417794612458684417), 228 (1417794612479655937), 229 (1417794612502724608), 230 (1417794612550959106), 233 (1417794612560396289)]} 0 150
[junit4:junit4]   2> 16305 T150 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[305 (1417794612478607360), 307 (1417794612486995968), 309 (1417794612502724609), 312 (1417794612507967489), 313 (1417794612510064641), 314 (1417794612515307522), 315 (1417794612517404672), 316 (1417794612543619073), 317 (1417794612546764801), 318 (1417794612555153408)]} 0 75
[junit4:junit4]   2> 16319 T139 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[234 (1417794612405207040), 235 (1417794612450295808), 238 (1417794612455538690), 242 (1417794612489093120), 244 (1417794612515307520), 246 (1417794612544667649), 247 (1417794612550959104), 248 (1417794612558299136), 250 (1417794612562493440), 251 (1417794612567736320)]} 0 159
[junit4:junit4]   2> 16320 T148 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[357 (1417794612497481728), 358 (1417794612503773184), 363 (1417794612513210368), 364 (1417794612517404673), 365 (1417794612542570496), 366 (1417794612546764800), 367 (1417794612549910528), 368 (1417794612553056256), 372 (1417794612556201985), 373 (1417794612566687744)]} 0 71
[junit4:junit4]   2> 16326 T51 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[390 (1417794612545716225), 391 (1417794612548861952), 392 (1417794612550959104), 393 (1417794612552007681), 394 (1417794612554104832), 395 (1417794612556201984), 396 (1417794612558299136), 397 (1417794612560396289), 398 (1417794612562493440), 399 (1417794612563542016)]} 0 33
[junit4:junit4]   2> 16325 T67 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[319 (1417794612479655936), 321 (1417794612481753089), 328 (1417794612507967488), 330 (1417794612510064640), 332 (1417794612515307521), 333 (1417794612544667648), 336 (1417794612552007681), 337 (1417794612557250560), 338 (1417794612571930625), 339 (1417794612575076353)]} 0 95
[junit4:junit4]   2> 16325 T158 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[351 (1417794612493287424), 354 (1417794612496433152), 357 (1417794612498530304), 359 (1417794612510064640), 360 (1417794612512161792), 362 (1417794612514258944), 363 (1417794612519501825), 365 (1417794612552007681), 366 (1417794612554104832), 369 (1417794612575076353)]} 0 81
[junit4:junit4]   2> 16319 T154 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[287 (1417794612482801665), 288 (1417794612489093120), 289 (1417794612497481728), 290 (1417794612503773186), 294 (1417794612506918912), 297 (1417794612546764800), 300 (1417794612550959106), 303 (1417794612556201985), 307 (1417794612559347712), 308 (1417794612566687744)]} 0 86
[junit4:junit4]   2> 16329 T149 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[285 (1417794612482801664), 288 (1417794612489093121), 291 (1417794612507967490), 296 (1417794612512161792), 297 (1417794612543619074), 299 (1417794612545716225), 301 (1417794612554104832), 302 (1417794612560396288), 303 (1417794612564590592), 304 (1417794612569833472)]} 0 95
[junit4:junit4]   2> 16329 T153 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[311 (1417794612488044544), 312 (1417794612493287424), 313 (1417794612503773184), 314 (1417794612550959104), 316 (1417794612553056257), 317 (1417794612556201986), 319 (1417794612564590593), 320 (1417794612567736321), 321 (1417794612575076353), 324 (1417794612578222080)]} 0 91
[junit4:junit4]   2> 16341 T146 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[374 (1417794612552007680), 377 (1417794612556201984), 379 (1417794612558299137), 382 (1417794612561444864), 386 (1417794612563542016), 390 (1417794612575076352), 391 (1417794612577173504), 397 (1417794612583464960), 399 (1417794612586610688), 402 (1417794612589756416)]} 0 42
[junit4:junit4]   2> 16343 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[383 (1417794612522647552), 385 (1417794612550959105), 386 (1417794612553056256), 387 (1417794612555153408), 388 (1417794612557250560), 390 (1417794612564590592), 391 (1417794612566687746), 395 (1417794612568784896), 396 (1417794612577173504), 398 (1417794612579270657)]} 0 72
[junit4:junit4]   2> 16346 T105 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[340 (1417794612493287424), 342 (1417794612497481729), 343 (1417794612543619072), 345 (1417794612545716224), 346 (1417794612553056257), 347 (1417794612569833473), 348 (1417794612586610689), 352 (1417794612589756417), 353 (1417794612594999297), 355 (1417794612597096450)]} 0 102
[junit4:junit4]   2> 16348 T112 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[404 (1417794612558299136), 405 (1417794612560396288), 406 (1417794612562493440), 410 (1417794612563542018), 413 (1417794612575076352), 414 (1417794612577173504), 415 (1417794612586610690), 419 (1417794612589756417), 420 (1417794612591853568), 421 (1417794612600242179)]} 0 43
[junit4:junit4]   2> 16349 T114 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[325 (1417794612479655938), 327 (1417794612482801664), 330 (1417794612488044545), 331 (1417794612503773185), 332 (1417794612518453248), 334 (1417794612560396288), 336 (1417794612576124928), 338 (1417794612592902145), 339 (1417794612597096448), 343 (1417794612601290753)]} 0 118
[junit4:junit4]   2> 16351 T128 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[326 (1417794612481753088), 327 (1417794612490141697), 331 (1417794612506918913), 332 (1417794612545716225), 335 (1417794612549910529), 340 (1417794612561444864), 341 (1417794612564590592), 346 (1417794612568784896), 349 (1417794612587659264), 350 (1417794612599193601)]} 0 119
[junit4:junit4]   2> 16353 T161 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[373 (1417794612545716224), 376 (1417794612548861953), 380 (1417794612552007680), 382 (1417794612554104833), 383 (1417794612557250560), 384 (1417794612560396288), 385 (1417794612569833472), 386 (1417794612582416385), 387 (1417794612592902145), 389 (1417794612603387904)]} 0 60
[junit4:junit4]   2> 16354 T129 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[420 (1417794612571930624), 421 (1417794612574027776), 422 (1417794612576124929), 423 (1417794612578222080), 427 (1417794612584513537), 428 (1417794612593950720), 431 (1417794612596047872), 433 (1417794612600242176), 435 (1417794612602339328), 437 (1417794612605485056)]} 0 35
[junit4:junit4]   2> 16354 T126 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[344 (1417794612512161792), 345 (1417794612515307520), 346 (1417794612543619072), 347 (1417794612548861952), 348 (1417794612554104832), 353 (1417794612562493440), 355 (1417794612570882048), 356 (1417794612593950721), 358 (1417794612597096449), 360 (1417794612605485057)]} 0 93
[junit4:junit4]   2> 16357 T51 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[481 (1417794612581367808), 482 (1417794612583464961), 483 (1417794612585562112), 484 (1417794612589756418), 485 (1417794612591853568), 486 (1417794612593950720), 488 (1417794612596047872), 490 (1417794612597096448), 491 (1417794612603387905), 492 (1417794612609679361)]} 0 29
[junit4:junit4]   2> 16360 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[363 (1417794612516356097), 367 (1417794612542570496), 370 (1417794612544667648), 372 (1417794612556201984), 375 (1417794612569833472), 376 (1417794612586610688), 377 (1417794612598145026), 379 (1417794612603387904), 380 (1417794612606533632), 381 (1417794612611776512)]} 0 94
[junit4:junit4]   2> 16365 T66 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[403 (1417794612559347712), 405 (1417794612584513538), 408 (1417794612587659264), 409 (1417794612590804993), 410 (1417794612593950722), 411 (1417794612597096449), 415 (1417794612609679361), 416 (1417794612612825088), 418 (1417794612614922241), 419 (1417794612618067968)]} 0 59
[junit4:junit4]   2> 16372 T65 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[461 (1417794612584513536), 462 (1417794612586610690), 463 (1417794612590804992), 464 (1417794612593950721), 465 (1417794612597096448), 470 (1417794612612825089), 476 (1417794612615970817), 477 (1417794612621213696), 481 (1417794612623310848), 483 (1417794612625408002)]} 0 42
[junit4:junit4]   2> 16372 T150 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[485 (1417794612594999296), 486 (1417794612608630784), 489 (1417794612609679362), 494 (1417794612611776513), 496 (1417794612613873665), 497 (1417794612615970816), 502 (1417794612620165120), 504 (1417794612621213697), 505 (1417794612623310849), 509 (1417794612625408001)]} 0 32
[junit4:junit4]   2> 16372 T125 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[309 (1417794612506918912), 310 (1417794612511113216), 311 (1417794612552007680), 312 (1417794612563542017), 318 (1417794612582416384), 319 (1417794612599193602), 321 (1417794612610727936), 322 (1417794612617019392), 323 (1417794612621213698), 325 (1417794612624359424)]} 0 151
[junit4:junit4]   2> 16373 T160 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[416 (1417794612565639168), 417 (1417794612582416384), 418 (1417794612584513537), 419 (1417794612589756417), 420 (1417794612590804992), 421 (1417794612601290752), 423 (1417794612609679360), 426 (1417794612621213696), 429 (1417794612624359424), 433 (1417794612626456576)]} 0 61
[junit4:junit4]   2> 16382 T166 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[482 (1417794612592902144), 483 (1417794612602339329), 484 (1417794612606533632), 485 (1417794612611776512), 489 (1417794612619116545), 491 (1417794612621213696), 492 (1417794612623310849), 494 (1417794612625408001), 495 (1417794612628553728), 496 (1417794612635893761)]} 0 43
[junit4:junit4]   2> 16382 T80 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[442 (1417794612581367808), 445 (1417794612582416385), 446 (1417794612586610688), 447 (1417794612600242177), 448 (1417794612602339328), 451 (1417794612609679360), 453 (1417794612614922240), 456 (1417794612619116549), 457 (1417794612633796609), 458 (1417794612635893760)]} 0 55
[junit4:junit4]   2> 16382 T148 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[439 (1417794612576124928), 444 (1417794612604436480), 445 (1417794612607582208), 447 (1417794612611776512), 450 (1417794612613873664), 452 (1417794612622262272), 454 (1417794612625408000), 455 (1417794612627505152), 457 (1417794612633796608), 459 (1417794612635893760)]} 0 60
[junit4:junit4]   2> 16383 T53 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[401 (1417794612559347712), 403 (1417794612564590592), 404 (1417794612571930624), 406 (1417794612580319232), 407 (1417794612584513536), 408 (1417794612592902144), 409 (1417794612598145024), 410 (1417794612609679363), 413 (1417794612613873664), 414 (1417794612618067968)]} 0 76
[junit4:junit4]   2> 16384 T122 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[494 (1417794612602339328), 496 (1417794612604436481), 497 (1417794612609679364), 498 (1417794612612825088), 499 (1417794612615970816), 500 (1417794612620165121), 501 (1417794612623310849), 504 (1417794612625408001), 505 (1417794612628553728), 506 (1417794612636942336)]} 0 39
[junit4:junit4]   2> 16385 T147 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[461 (1417794612583464960), 462 (1417794612586610688), 463 (1417794612594999296), 465 (1417794612596047873), 469 (1417794612601290753), 474 (1417794612604436482), 475 (1417794612609679362), 477 (1417794612614922240), 479 (1417794612629602304), 480 (1417794612639039489)]} 0 59
[junit4:junit4]   2> 16669 T81 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[535 (1417794612923203584), 536 (1417794612924252160), 537 (1417794612926349312), 540 (1417794612927397888), 544 (1417794612929495040), 548 (1417794612930543616), 549 (1417794612932640768), 550 (1417794612933689344), 552 (1417794612935786496), 553 (1417794612936835072)]} 0 15
[junit4:junit4]   2> 16720 T153 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[493 (1417794612607582208), 494 (1417794612613873665), 496 (1417794612614922240), 498 (1417794612622262272), 500 (1417794612624359424), 501 (1417794612626456576), 502 (1417794612628553728), 503 (1417794612629602304), 504 (1417794612640088065), 505 (1417794612642185217)]} 0 368
[junit4:junit4]   2> 16722 T139 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[511 (1417794612596047873), 512 (1417794612603387904), 513 (1417794612606533632), 515 (1417794612609679360), 517 (1417794612614922240), 519 (1417794612620165121), 520 (1417794612629602304), 521 (1417794612636942336), 522 (1417794612641136640), 523 (1417794612920057856)]} 0 380
[junit4:junit4]   2> 16728 T165 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[422 (1417794612566687744), 423 (1417794612585562112), 425 (1417794612600242178), 426 (1417794612608630784), 427 (1417794612612825088), 428 (1417794612619116547), 429 (1417794612623310848), 432 (1417794612628553729), 434 (1417794612639039488), 435 (1417794612642185216)]} 0 414
[junit4:junit4]   2> 16729 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[435 (1417794612567736320), 436 (1417794612569833473), 437 (1417794612572979200), 438 (1417794612578222081), 439 (1417794612593950723), 441 (1417794612598145024), 442 (1417794612602339328), 443 (1417794612609679361), 446 (1417794612640088067), 448 (1417794612990312449)]} 0 415
[junit4:junit4]   2> 16729 T136 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[384 (1417794612550959104), 388 (1417794612563542016), 392 (1417794612570882048), 394 (1417794612576124928), 397 (1417794612586610689), 398 (1417794612600242176), 399 (1417794612619116548), 401 (1417794612621213699), 402 (1417794612625408002), 403 (1417794612993458176)]} 0 431
[junit4:junit4]   2> 16726 T164 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[497 (1417794612589756416), 499 (1417794612590804992), 500 (1417794612599193600), 503 (1417794612601290752), 505 (1417794612605485056), 507 (1417794612608630785), 511 (1417794612611776513), 512 (1417794612621213697), 513 (1417794612625408000), 515 (1417794612640088064)]} 0 390
[junit4:junit4]   2> 16730 T154 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[466 (1417794612586610689), 470 (1417794612593950722), 471 (1417794612601290752), 475 (1417794612604436480), 476 (1417794612607582209), 482 (1417794612612825088), 484 (1417794612628553729), 488 (1417794612640088064), 489 (1417794612990312450), 490 (1417794612994506754)]} 0 400
[junit4:junit4]   2> 16734 T108 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[401 (1417794612565639168), 403 (1417794612571930625), 405 (1417794612575076352), 407 (1417794612579270656), 408 (1417794612592902146), 409 (1417794612598145025), 413 (1417794612606533633), 414 (1417794612609679360), 415 (1417794612637990912), 416 (1417794612997652480)]} 0 421
[junit4:junit4]   2> 16735 T106 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[419 (1417794612560396289), 420 (1417794612563542016), 424 (1417794612566687745), 428 (1417794612571930624), 429 (1417794612577173505), 430 (1417794612592902144), 431 (1417794612617019392), 432 (1417794612640088066), 433 (1417794612994506753), 434 (1417794613006041088)]} 0 427
[junit4:junit4]   2> 16735 T114 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[508 (1417794612623310848), 509 (1417794612625408000), 512 (1417794612627505152), 515 (1417794612632748032), 517 (1417794612635893760), 519 (1417794612637990913), 521 (1417794612642185216), 523 (1417794612994506752), 525 (1417794613002895361), 529 (1417794613006041090)]} 0 368
[junit4:junit4]   2> 16736 T120 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[449 (1417794612577173506), 452 (1417794612593950720), 453 (1417794612601290754), 454 (1417794612605485056), 457 (1417794612606533634), 458 (1417794612613873664), 462 (1417794612990312448), 463 (1417794612993458176), 464 (1417794613002895360), 465 (1417794613006041089)]} 0 412
[junit4:junit4]   2> 16742 T100 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[516 (1417794612619116546), 517 (1417794612622262272), 518 (1417794612624359425), 519 (1417794612627505152), 521 (1417794612633796608), 527 (1417794612636942336), 530 (1417794612995555328), 531 (1417794613001846784), 532 (1417794613006041088), 533 (1417794613008138240)]} 0 379
[junit4:junit4]   2> 16743 T52 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[509 (1417794612620165120), 511 (1417794612623310848), 512 (1417794612625408000), 513 (1417794612635893760), 517 (1417794612639039488), 518 (1417794612920057856), 519 (1417794612992409601), 520 (1417794613003943936), 521 (1417794613009186816), 524 (1417794613012332544)]} 0 378
[junit4:junit4]   2> 16739 T151 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[434 (1417794612589756416), 437 (1417794612604436480), 438 (1417794612608630784), 439 (1417794612611776512), 444 (1417794612615970817), 446 (1417794612619116544), 447 (1417794612640088064), 449 (1417794612991361024), 458 (1417794612998701056), 459 (1417794613002895361)]} 0 404
[junit4:junit4]   2> 16752 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[573 (1417794613007089665), 575 (1417794613009186816), 576 (1417794613011283969), 579 (1417794613012332545), 581 (1417794613014429696), 583 (1417794613016526848), 586 (1417794613017575424), 587 (1417794613018624001), 588 (1417794613020721152), 589 (1417794613021769728)]} 0 18
[junit4:junit4]   2> 16755 T103 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[526 (1417794612642185216), 528 (1417794612992409600), 529 (1417794612994506752), 531 (1417794613000798208), 532 (1417794613002895362), 533 (1417794613007089664), 535 (1417794613010235392), 536 (1417794613012332545), 539 (1417794613025964033), 544 (1417794613028061184)]} 0 370
[junit4:junit4]   2> 16758 T102 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[524 (1417794612922155008), 525 (1417794612996603904), 527 (1417794612999749632), 529 (1417794613001846784), 536 (1417794613007089664), 537 (1417794613009186816), 541 (1417794613012332544), 542 (1417794613022818304), 544 (1417794613027012608), 545 (1417794613029109762)]} 0 106
[junit4:junit4]   2> 16760 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[590 (1417794613008138241), 591 (1417794613011283968), 593 (1417794613012332544), 596 (1417794613013381121), 597 (1417794613022818304), 603 (1417794613023866880), 605 (1417794613024915456), 607 (1417794613028061184), 608 (1417794613030158336), 612 (1417794613032255488)]} 0 25
[junit4:junit4]   2> 16762 T78 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[463 (1417794612581367809), 466 (1417794612591853569), 472 (1417794612599193603), 473 (1417794612619116544), 474 (1417794612623310850), 475 (1417794612920057856), 476 (1417794613000798208), 477 (1417794613003943936), 478 (1417794613016526848), 481 (1417794613018624001)]} 0 434
[junit4:junit4]   2> 16808 T105 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[546 (1417794613001846785), 548 (1417794613006041088), 551 (1417794613008138240), 553 (1417794613011283968), 555 (1417794613021769728), 557 (1417794613024915456), 559 (1417794613029109760), 564 (1417794613031206914), 567 (1417794613035401216), 568 (1417794613082587136)]} 0 78
[junit4:junit4]   2> 16818 T66 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[610 (1417794613027012609), 611 (1417794613029109761), 612 (1417794613031206913), 613 (1417794613034352641), 617 (1417794613036449793), 623 (1417794613082587137), 629 (1417794613085732864), 632 (1417794613087830016), 634 (1417794613089927168), 636 (1417794613092024320)]} 0 65
[junit4:junit4]   2> 16820 T101 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[545 (1417794612999749632), 547 (1417794613002895360), 548 (1417794613011283968), 553 (1417794613014429696), 555 (1417794613017575424), 556 (1417794613018624000), 559 (1417794613020721152), 562 (1417794613025964032), 563 (1417794613029109760), 565 (1417794613095170048)]} 0 93
[junit4:junit4]   2> 16821 T129 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[569 (1417794613010235392), 572 (1417794613012332545), 574 (1417794613015478272), 575 (1417794613017575424), 577 (1417794613019672576), 580 (1417794613031206912), 581 (1417794613034352640), 586 (1417794613036449792), 589 (1417794613095170048), 591 (1417794613096218624)]} 0 84
[junit4:junit4]   2> 16822 T51 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[566 (1417794613027012608), 567 (1417794613029109761), 569 (1417794613032255488), 571 (1417794613081538560), 576 (1417794613083635712), 577 (1417794613084684289), 579 (1417794613086781440), 580 (1417794613087830017), 581 (1417794613089927168), 582 (1417794613092024320)]} 0 70
[junit4:junit4]   2> 16823 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[533 (1417794613001846784), 536 (1417794613006041091), 540 (1417794613008138240), 544 (1417794613010235392), 547 (1417794613016526849), 548 (1417794613018624000), 549 (1417794613021769729), 550 (1417794613023866881), 551 (1417794613030158337), 553 (1417794613034352641)]} 0 34
[junit4:junit4]   2> 16824 T122 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[601 (1417794613030158336), 603 (1417794613032255489), 605 (1417794613082587136), 608 (1417794613084684288), 609 (1417794613085732864), 611 (1417794613087830016), 613 (1417794613089927169), 616 (1417794613092024321), 617 (1417794613093072896), 619 (1417794613099364352)]} 0 68
[junit4:junit4]   2> 16899 T153 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[554 (1417794613007089664), 555 (1417794613009186817), 556 (1417794613011283970), 558 (1417794613013381120), 559 (1417794613022818305), 560 (1417794613029109760), 561 (1417794613032255489), 564 (1417794613035401216), 567 (1417794613101461504), 571 (1417794613102510081)]} 0 166
[junit4:junit4]   2> 16901 T100 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[575 (1417794613018624000), 576 (1417794613020721152), 577 (1417794613034352640), 578 (1417794613036449793), 579 (1417794613083635712), 583 (1417794613085732865), 584 (1417794613088878593), 585 (1417794613090975745), 588 (1417794613093072897), 589 (1417794613100412928)]} 0 157
[junit4:junit4]   2> 16906 T80 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[606 (1417794613081538560), 611 (1417794613084684288), 613 (1417794613086781440), 614 (1417794613088878592), 615 (1417794613090975744), 617 (1417794613093072896), 618 (1417794613099364352), 619 (1417794613101461504), 621 (1417794613102510082), 622 (1417794613180104704)]} 0 144
[junit4:junit4]   2> 16913 T145 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[583 (1417794613027012609), 586 (1417794613030158337), 587 (1417794613035401216), 589 (1417794613096218624), 591 (1417794613101461504), 595 (1417794613102510080), 596 (1417794613180104704), 597 (1417794613184299008), 599 (1417794613188493312), 600 (1417794613191639040)]} 0 160
[junit4:junit4]   2> 16917 T158 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[554 (1417794613013381120), 555 (1417794613024915456), 556 (1417794613029109760), 559 (1417794613031206912), 563 (1417794613036449792), 564 (1417794613102510080), 565 (1417794613187444736), 567 (1417794613190590464), 568 (1417794613192687616), 569 (1417794613193736192)]} 0 177
[junit4:junit4]   2> 16926 T150 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[592 (1417794613179056128), 593 (1417794613184299008), 594 (1417794613192687617), 595 (1417794613194784769), 598 (1417794613200027648), 600 (1417794613201076224), 604 (1417794613203173376), 606 (1417794613204221952), 608 (1417794613205270529), 609 (1417794613207367680)]} 0 101
[junit4:junit4]   2> 16932 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[697 (1417794613203173376), 698 (1417794613204221952), 700 (1417794613205270528), 703 (1417794613206319104), 704 (1417794613207367680), 705 (1417794613208416256), 706 (1417794613210513408), 707 (1417794613211561984), 710 (1417794613212610560), 711 (1417794613213659136)]} 0 12
[junit4:junit4]   2> 16933 T139 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[673 (1417794613200027649), 674 (1417794613201076225), 675 (1417794613203173377), 683 (1417794613205270528), 684 (1417794613206319104), 685 (1417794613208416256), 686 (1417794613209464832), 690 (1417794613211561984), 695 (1417794613212610560), 703 (1417794613214707712)]} 0 16
[junit4:junit4]   2> 16934 T147 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[620 (1417794613101461505), 622 (1417794613187444736), 624 (1417794613190590464), 625 (1417794613193736192), 626 (1417794613200027649), 630 (1417794613202124801), 634 (1417794613205270529), 637 (1417794613208416257), 638 (1417794613211561984), 641 (1417794613213659137)]} 0 172
[junit4:junit4]   2> 16935 T166 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[593 (1417794613027012608), 594 (1417794613035401216), 595 (1417794613082587136), 596 (1417794613085732864), 597 (1417794613087830016), 598 (1417794613089927168), 599 (1417794613092024320), 601 (1417794613099364353), 602 (1417794613102510081), 604 (1417794613186396160)]} 0 182
[junit4:junit4]   2> 16938 T128 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[757 (1417794613200027649), 758 (1417794613202124800), 759 (1417794613203173377), 764 (1417794613205270528), 766 (1417794613207367680), 768 (1417794613209464832), 772 (1417794613211561984), 773 (1417794613212610562), 775 (1417794613214707713), 776 (1417794613216804864)]} 0 20
[junit4:junit4]   2> 16939 T125 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[693 (1417794613203173376), 694 (1417794613204221953), 695 (1417794613206319105), 697 (1417794613208416257), 698 (1417794613210513409), 699 (1417794613212610560), 700 (1417794613214707712), 701 (1417794613215756289), 704 (1417794613217853441), 707 (1417794613219950592)]} 0 19
[junit4:junit4]   2> 16940 T161 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[665 (1417794613198979072), 667 (1417794613201076225), 671 (1417794613203173376), 675 (1417794613206319104), 678 (1417794613208416256), 680 (1417794613210513408), 682 (1417794613213659136), 684 (1417794613216804864), 686 (1417794613218902016), 688 (1417794613220999168)]} 0 25
[junit4:junit4]   2> 16940 T52 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[649 (1417794613200027648), 650 (1417794613202124800), 651 (1417794613205270528), 653 (1417794613207367680), 656 (1417794613209464832), 657 (1417794613212610560), 659 (1417794613214707712), 660 (1417794613217853441), 661 (1417794613219950592), 663 (1417794613220999169)]} 0 23
[junit4:junit4]   2> 16941 T112 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[778 (1417794613201076224), 779 (1417794613204221952), 780 (1417794613206319104), 781 (1417794613208416256), 782 (1417794613210513408), 784 (1417794613212610561), 785 (1417794613215756288), 787 (1417794613217853440), 788 (1417794613220999168), 789 (1417794613222047744)]} 0 22
[junit4:junit4]   2> 16952 T148 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[655 (1417794613186396160), 656 (1417794613187444736), 658 (1417794613189541888), 661 (1417794613190590464), 663 (1417794613192687616), 664 (1417794613194784768), 666 (1417794613227290624), 667 (1417794613230436352), 669 (1417794613232533504), 670 (1417794613233582082)]} 0 51
[junit4:junit4]   2> 16955 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[644 (1417794613180104704), 646 (1417794613184299008), 651 (1417794613185347585), 652 (1417794613192687617), 653 (1417794613194784768), 654 (1417794613227290625), 656 (1417794613232533504), 657 (1417794613234630656), 659 (1417794613235679235), 661 (1417794613237776384)]} 0 57
[junit4:junit4]   2> 16956 T117 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[727 (1417794613183250432), 729 (1417794613185347584), 732 (1417794613192687616), 737 (1417794613216804865), 739 (1417794613219950593), 740 (1417794613223096320), 741 (1417794613225193473), 742 (1417794613227290625), 747 (1417794613234630656), 750 (1417794613236727809)]} 0 54
[junit4:junit4]   2> 16958 T154 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[614 (1417794613034352640), 615 (1417794613100412928), 617 (1417794613187444736), 619 (1417794613189541888), 620 (1417794613191639041), 622 (1417794613219950593), 623 (1417794613222047745), 624 (1417794613225193472), 627 (1417794613227290624), 628 (1417794613235679232)]} 0 198
[junit4:junit4]   2> 16958 T160 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[753 (1417794613201076224), 757 (1417794613204221952), 759 (1417794613207367681), 761 (1417794613209464833), 763 (1417794613212610561), 764 (1417794613215756288), 765 (1417794613217853440), 766 (1417794613225193472), 767 (1417794613227290624), 768 (1417794613237776384)]} 0 40
[junit4:junit4]   2> 16962 T114 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[629 (1417794613191639040), 630 (1417794613194784769), 631 (1417794613228339201), 634 (1417794613235679233), 636 (1417794613236727808), 637 (1417794613237776386), 638 (1417794613239873536), 639 (1417794613240922113), 641 (1417794613244067840), 642 (1417794613245116416)]} 0 60
[junit4:junit4]   2> 16971 T133 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[647 (1417794613184299008), 648 (1417794613189541889), 651 (1417794613228339202), 652 (1417794613233582080), 653 (1417794613237776384), 657 (1417794613238824960), 658 (1417794613240922112), 661 (1417794613244067840), 662 (1417794613247213568), 664 (1417794613251407872)]} 0 69
[junit4:junit4]   2> 16971 T165 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[624 (1417794613095170048), 626 (1417794613097267200), 628 (1417794613178007552), 630 (1417794613184299009), 631 (1417794613195833344), 634 (1417794613218902016), 635 (1417794613222047745), 636 (1417794613224144896), 639 (1417794613226242048), 641 (1417794613228339201)]} 0 153
[junit4:junit4]   2> 16979 T108 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[663 (1417794613082587136), 664 (1417794613102510080), 666 (1417794613179056128), 667 (1417794613183250433), 668 (1417794613185347584), 669 (1417794613192687616), 671 (1417794613193736192), 672 (1417794613195833344), 673 (1417794613255602176), 674 (1417794613256650752)]} 0 174
[junit4:junit4]   2> 16980 T146 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[637 (1417794613230436353), 638 (1417794613241970690), 639 (1417794613245116417), 640 (1417794613246164993), 645 (1417794613253505024), 646 (1417794613255602176), 647 (1417794613257699328), 648 (1417794613258747904), 649 (1417794613262942208), 651 (1417794613263990785)]} 0 34
[junit4:junit4]   2> 16981 T67 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[726 (1417794613233582081), 729 (1417794613236727809), 730 (1417794613237776384), 733 (1417794613239873536), 735 (1417794613241970689), 738 (1417794613245116416), 739 (1417794613247213568), 740 (1417794613255602177), 741 (1417794613259796481), 742 (1417794613263990784)]} 0 31
[junit4:junit4]   2> 16979 T120 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[714 (1417794613194784770), 716 (1417794613217853441), 717 (1417794613219950592), 718 (1417794613222047744), 719 (1417794613223096320), 720 (1417794613225193473), 722 (1417794613226242049), 725 (1417794613233582080), 727 (1417794613240922114), 728 (1417794613258747905)]} 0 66
[junit4:junit4]   2> 16983 T132 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[665 (1417794613200027648), 670 (1417794613218902017), 673 (1417794613220999169), 674 (1417794613222047746), 678 (1417794613225193472), 679 (1417794613227290624), 682 (1417794613228339200), 685 (1417794613255602176), 686 (1417794613258747904), 690 (1417794613265039360)]} 0 66
[junit4:junit4]   2> 16983 T151 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[769 (1417794613233582080), 770 (1417794613235679232), 771 (1417794613236727808), 774 (1417794613237776385), 776 (1417794613240922113), 777 (1417794613241970688), 780 (1417794613245116417), 781 (1417794613247213568), 782 (1417794613256650752), 784 (1417794613265039361)]} 0 33
[junit4:junit4]   2> 16984 T65 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[705 (1417794613232533505), 707 (1417794613236727808), 708 (1417794613238824960), 710 (1417794613240922112), 713 (1417794613249310720), 717 (1417794613251407872), 719 (1417794613252456448), 720 (1417794613254553600), 721 (1417794613258747905), 723 (1417794613265039360)]} 0 36
[junit4:junit4]   2> 16984 T164 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[724 (1417794613189541888), 725 (1417794613190590465), 726 (1417794613234630656), 727 (1417794613236727808), 728 (1417794613237776385), 730 (1417794613247213569), 732 (1417794613250359297), 733 (1417794613252456448), 737 (1417794613256650752), 738 (1417794613262942209)]} 0 76
[junit4:junit4]   2> 16987 T136 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[739 (1417794613235679232), 740 (1417794613239873536), 741 (1417794613241970688), 744 (1417794613246164992), 745 (1417794613250359296), 746 (1417794613252456449), 749 (1417794613254553600), 752 (1417794613257699328), 754 (1417794613265039362), 756 (1417794613270282240)]} 0 36
[junit4:junit4]   2> 16999 T149 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[746 (1417794613233582080), 748 (1417794613236727810), 749 (1417794613239873537), 752 (1417794613241970688), 754 (1417794613244067840), 756 (1417794613246164992), 760 (1417794613254553601), 763 (1417794613259796480), 764 (1417794613263990786), 767 (1417794613266087937)]} 0 49
[junit4:junit4]   2> 17001 T106 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[677 (1417794613183250432), 680 (1417794613188493312), 683 (1417794613190590464), 687 (1417794613217853440), 688 (1417794613220999168), 689 (1417794613224144896), 691 (1417794613226242048), 693 (1417794613228339200), 694 (1417794613267136514), 696 (1417794613271330818)]} 0 102
[junit4:junit4]   2> 17002 T126 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[754 (1417794613235679234), 756 (1417794613246164992), 759 (1417794613247213568), 762 (1417794613250359297), 769 (1417794613252456448), 770 (1417794613255602177), 773 (1417794613258747904), 775 (1417794613262942208), 776 (1417794613265039360), 781 (1417794613271330816)]} 0 50
[junit4:junit4]   2> 17003 T103 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[785 (1417794613252456448), 788 (1417794613254553600), 790 (1417794613262942208), 792 (1417794613263990784), 793 (1417794613266087936), 796 (1417794613267136514), 798 (1417794613270282240), 800 (1417794613272379392), 801 (1417794613284962304), 803 (1417794613287059456)]} 0 35
[junit4:junit4]   2> 17002 T153 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[734 (1417794613230436352), 739 (1417794613237776385), 740 (1417794613240922112), 741 (1417794613250359296), 743 (1417794613257699328), 744 (1417794613259796480), 747 (1417794613263990785), 749 (1417794613267136512), 750 (1417794613270282241), 752 (1417794613272379393)]} 0 55
[junit4:junit4]   2> 17005 T81 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[708 (1417794613256650753), 709 (1417794613262942208), 710 (1417794613263990784), 712 (1417794613265039361), 714 (1417794613267136512), 715 (1417794613269233664), 716 (1417794613271330817), 718 (1417794613272379392), 720 (1417794613284962304), 722 (1417794613286010880)]} 0 45
[junit4:junit4]   2> 17006 T53 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[708 (1417794613240922112), 710 (1417794613245116416), 714 (1417794613246164992), 716 (1417794613253505024), 719 (1417794613260845056), 720 (1417794613265039360), 722 (1417794613267136513), 723 (1417794613271330816), 724 (1417794613286010880), 725 (1417794613289156608)]} 0 50
[junit4:junit4]   2> 17005 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[782 (1417794613251407872), 785 (1417794613253505024), 786 (1417794613258747906), 787 (1417794613266087936), 789 (1417794613268185088), 791 (1417794613270282240), 792 (1417794613283913728), 793 (1417794613287059456), 794 (1417794613289156609), 795 (1417794613290205184)]} 0 38
[junit4:junit4]   2> 17007 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[796 (1417794613263990784), 797 (1417794613267136513), 798 (1417794613269233664), 799 (1417794613272379392), 801 (1417794613284962304), 804 (1417794613286010881), 806 (1417794613287059458), 809 (1417794613289156610), 813 (1417794613290205186), 816 (1417794613292302337)]} 0 29
[junit4:junit4]   2> 17010 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[817 (1417794613267136515), 821 (1417794613271330817), 822 (1417794613283913729), 824 (1417794613286010880), 826 (1417794613287059457), 828 (1417794613289156608), 829 (1417794613290205185), 830 (1417794613292302336), 831 (1417794613293350912), 832 (1417794613294399488)]} 0 29
[junit4:junit4]   2> 17011 T129 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[799 (1417794613265039361), 800 (1417794613269233664), 801 (1417794613271330816), 804 (1417794613284962304), 808 (1417794613287059456), 809 (1417794613289156608), 811 (1417794613291253761), 812 (1417794613292302337), 814 (1417794613294399488), 823 (1417794613295448065)]} 0 30
[junit4:junit4]   2> 17011 T127 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[689 (1417794613231484928), 690 (1417794613232533504), 695 (1417794613244067840), 696 (1417794613257699328), 697 (1417794613267136512), 698 (1417794613268185088), 702 (1417794613272379393), 705 (1417794613286010881), 706 (1417794613293350912), 707 (1417794613295448065)]} 0 64
[junit4:junit4]   2> 17012 T102 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[768 (1417794613262942209), 777 (1417794613266087936), 779 (1417794613268185088), 784 (1417794613288108032), 786 (1417794613289156609), 788 (1417794613291253760), 789 (1417794613292302336), 790 (1417794613293350912), 792 (1417794613295448064), 795 (1417794613296496640)]} 0 34
[junit4:junit4]   2> 17018 T78 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[790 (1417794613262942210), 791 (1417794613271330816), 796 (1417794613284962305), 797 (1417794613287059456), 798 (1417794613289156608), 800 (1417794613291253760), 806 (1417794613294399488), 811 (1417794613296496640), 812 (1417794613299642368), 817 (1417794613301739520)]} 0 40
[junit4:junit4]   2> 17030 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[833 (1417794613295448064), 837 (1417794613297545216), 843 (1417794613298593792), 844 (1417794613304885249), 845 (1417794613306982401), 848 (1417794613309079552), 852 (1417794613310128129), 855 (1417794613312225280), 857 (1417794613313273856), 858 (1417794613315371008)]} 0 21
[junit4:junit4]   2> 17032 T105 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[849 (1417794613297545216), 850 (1417794613299642368), 855 (1417794613300690944), 857 (1417794613302788096), 859 (1417794613303836672), 860 (1417794613305933824), 863 (1417794613309079554), 864 (1417794613311176704), 867 (1417794613313273856), 871 (1417794613315371008)]} 0 22
[junit4:junit4]   2> 17042 T158 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[836 (1417794613300690944), 839 (1417794613303836672), 840 (1417794613305933825), 843 (1417794613314322432), 844 (1417794613317468160), 845 (1417794613319565312), 849 (1417794613321662466), 850 (1417794613323759617), 851 (1417794613325856769), 852 (1417794613327953921)]} 0 29
[junit4:junit4]   2> 17043 T114 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[859 (1417794613299642368), 863 (1417794613300690944), 867 (1417794613302788096), 870 (1417794613303836672), 871 (1417794613305933824), 873 (1417794613309079553), 874 (1417794613311176704), 875 (1417794613312225281), 876 (1417794613325856768), 878 (1417794613327953920)]} 0 31
[junit4:junit4]   2> 17046 T101 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[804 (1417794613300690945), 805 (1417794613305933824), 806 (1417794613308030976), 809 (1417794613314322433), 812 (1417794613318516736), 814 (1417794613321662464), 815 (1417794613324808192), 816 (1417794613326905344), 817 (1417794613329002496), 818 (1417794613332148224)]} 0 33
[junit4:junit4]   2> 17047 T127 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[819 (1417794613300690944), 820 (1417794613302788096), 823 (1417794613305933825), 827 (1417794613309079552), 828 (1417794613317468162), 829 (1417794613319565312), 830 (1417794613322711041), 832 (1417794613324808194), 833 (1417794613327953921), 837 (1417794613332148225)]} 0 34
[junit4:junit4]   2> 17057 T66 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[825 (1417794613295448066), 827 (1417794613297545217), 831 (1417794613299642370), 835 (1417794613301739520), 838 (1417794613304885248), 839 (1417794613306982400), 841 (1417794613309079553), 842 (1417794613334245376), 846 (1417794613340536833), 847 (1417794613342633985)]} 0 48
[junit4:junit4]   2> 17059 T126 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[930 (1417794613317468160), 931 (1417794613319565312), 935 (1417794613320613888), 936 (1417794613322711040), 938 (1417794613332148225), 939 (1417794613334245377), 940 (1417794613341585409), 942 (1417794613343682560), 943 (1417794613344731137), 945 (1417794613345779713)]} 0 29
[junit4:junit4]   2> 17060 T161 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[904 (1417794613317468160), 906 (1417794613320613888), 908 (1417794613322711040), 909 (1417794613326905345), 912 (1417794613330051072), 922 (1417794613335293952), 923 (1417794613337391104), 925 (1417794613339488256), 926 (1417794613341585408), 928 (1417794613343682561)]} 0 30
[junit4:junit4]   2> 17061 T146 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[920 (1417794613320613888), 921 (1417794613323759616), 922 (1417794613324808192), 923 (1417794613326905344), 924 (1417794613327953920), 925 (1417794613330051072), 927 (1417794613335293952), 928 (1417794613336342528), 931 (1417794613337391104), 932 (1417794613339488257)]} 0 28
[junit4:junit4]   2> 17062 T112 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[916 (1417794613318516736), 917 (1417794613321662465), 920 (1417794613323759616), 922 (1417794613325856768), 923 (1417794613327953920), 929 (1417794613330051072), 930 (1417794613332148224), 932 (1417794613334245376), 933 (1417794613342633984), 938 (1417794613344731136)]} 0 32
[junit4:junit4]   2> 17062 T51 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[843 (1417794613295448064), 844 (1417794613297545216), 845 (1417794613299642368), 850 (1417794613301739520), 851 (1417794613306982400), 853 (1417794613311176704), 854 (1417794613314322432), 855 (1417794613316419584), 862 (1417794613340536832), 863 (1417794613347876865)]} 0 54
[junit4:junit4]   2> 17064 T153 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[946 (1417794613323759616), 947 (1417794613324808192), 948 (1417794613330051072), 950 (1417794613332148224), 951 (1417794613339488256), 952 (1417794613340536832), 953 (1417794613342633984), 954 (1417794613347876864), 955 (1417794613349974016), 956 (1417794613351022593)]} 0 29
[junit4:junit4]   2> 17066 T150 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[906 (1417794613323759617), 907 (1417794613325856768), 908 (1417794613329002496), 909 (1417794613331099648), 910 (1417794613333196800), 911 (1417794613341585409), 912 (1417794613344731137), 913 (1417794613346828290), 914 (1417794613348925440), 918 (1417794613351022593)]} 0 39
[junit4:junit4]   2> 17067 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[957 (1417794613334245376), 958 (1417794613341585408), 959 (1417794613344731136), 960 (1417794613345779712), 962 (1417794613346828291), 963 (1417794613348925440), 965 (1417794613349974017), 966 (1417794613352071168), 967 (1417794613353119746), 968 (1417794613355216897)]} 0 22
[junit4:junit4]   2> 17069 T108 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[879 (1417794613301739520), 880 (1417794613304885248), 881 (1417794613306982400), 883 (1417794613310128128), 884 (1417794613316419584), 885 (1417794613343682561), 888 (1417794613346828288), 892 (1417794613351022592), 893 (1417794613355216896), 896 (1417794613356265473)]} 0 54
[junit4:junit4]   2> 17070 T117 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[955 (1417794613335293953), 957 (1417794613338439680), 959 (1417794613340536833), 961 (1417794613342633985), 962 (1417794613344731136), 964 (1417794613346828288), 966 (1417794613348925441), 972 (1417794613354168320), 973 (1417794613356265473), 975 (1417794613357314050)]} 0 23
[junit4:junit4]   2> 17070 T122 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[866 (1417794613303836672), 870 (1417794613306982401), 872 (1417794613310128128), 873 (1417794613339488258), 874 (1417794613343682560), 876 (1417794613345779712), 877 (1417794613348925440), 878 (1417794613352071169), 881 (1417794613354168321), 882 (1417794613357314048)]} 0 53
[junit4:junit4]   2> 17074 T148 C3 UPDATE [collection1] webapp=/solr path

[...truncated too long message...]

 at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  24) Thread[id=54, name=qtp1989367070-54 Acceptor3 SelectChannelConnector@0.0.0.0:16252, state=BLOCKED, group=TGRP-TestDistributedSearch]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  25) Thread[id=36, name=qtp1286237868-36 Acceptor2 SelectChannelConnector@0.0.0.0:60522, state=RUNNABLE, group=TGRP-TestDistributedSearch]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  26) Thread[id=141, name=qtp536825004-141 Acceptor1 SelectChannelConnector@0.0.0.0:16272, state=BLOCKED, group=TGRP-TestDistributedSearch]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  27) Thread[id=68, name=qtp629076881-68 Acceptor3 SelectChannelConnector@0.0.0.0:16255, state=BLOCKED, group=TGRP-TestDistributedSearch]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  28) Thread[id=78, name=qtp736059716-78 Acceptor0 SelectChannelConnector@0.0.0.0:16256, state=RUNNABLE, group=TGRP-TestDistributedSearch]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  29) Thread[id=102, name=qtp779071350-102 Acceptor2 SelectChannelConnector@0.0.0.0:16267, state=RUNNABLE, group=TGRP-TestDistributedSearch]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  30) Thread[id=53, name=qtp1989367070-53 Acceptor2 SelectChannelConnector@0.0.0.0:16252, state=RUNNABLE, group=TGRP-TestDistributedSearch]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  31) Thread[id=51, name=qtp1989367070-51 Acceptor0 SelectChannelConnector@0.0.0.0:16252, state=BLOCKED, group=TGRP-TestDistributedSearch]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  32) Thread[id=23, name=qtp1412500084-23 Acceptor0 SelectChannelConnector@0.0.0.0:28646, state=BLOCKED, group=TGRP-TestDistributedSearch]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Compressing(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=408)), sim=DefaultSimilarity, locale=ar_YE, timezone=Europe/Budapest
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=33,free=100213968,total=125829120
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestSolrXMLSerializer, TestDistributedSearch]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDistributedSearch -Dtests.seed=43BDC8EDB8C9CCE8 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar_YE -Dtests.timezone=Europe/Budapest -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] ERROR   0.00s J1 | TestDistributedSearch (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([43BDC8EDB8C9CCE8]:0)
[junit4:junit4] Completed on J1 in 7224.04s, 1 test, 2 errors <<< FAILURES!

[...truncated 183 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:335: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/module-build.xml:61: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:1163: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:827: There were test failures: 233 suites, 965 tests, 3 suite-level errors, 1 error, 879 ignored

Total time: 138 minutes 25 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure

[JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 610 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.

Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/610/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.RecoveryZkTest

Error Message:
8 threads leaked from SUITE scope at org.apache.solr.cloud.RecoveryZkTest:     1) Thread[id=27, name=qtp806943770-27 Acceptor0 SelectChannelConnector@0.0.0.0:19540, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=30, name=qtp806943770-30 Acceptor3 SelectChannelConnector@0.0.0.0:19540, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=104, name=qtp1442633686-104 Acceptor1 SelectChannelConnector@0.0.0.0:17211, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=105, name=qtp1442633686-105 Acceptor2 SelectChannelConnector@0.0.0.0:17211, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=106, name=qtp1442633686-106 Acceptor3 SelectChannelConnector@0.0.0.0:17211, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=103, name=qtp1442633686-103 Acceptor0 SelectChannelConnector@0.0.0.0:17211, state=RUNNABLE, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=28, name=qtp806943770-28 Acceptor1 SelectChannelConnector@0.0.0.0:19540, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=29, name=qtp806943770-29 Acceptor2 SelectChannelConnector@0.0.0.0:19540, state=RUNNABLE, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 8 threads leaked from SUITE scope at org.apache.solr.cloud.RecoveryZkTest: 
   1) Thread[id=27, name=qtp806943770-27 Acceptor0 SelectChannelConnector@0.0.0.0:19540, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=30, name=qtp806943770-30 Acceptor3 SelectChannelConnector@0.0.0.0:19540, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=104, name=qtp1442633686-104 Acceptor1 SelectChannelConnector@0.0.0.0:17211, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=105, name=qtp1442633686-105 Acceptor2 SelectChannelConnector@0.0.0.0:17211, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=106, name=qtp1442633686-106 Acceptor3 SelectChannelConnector@0.0.0.0:17211, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=103, name=qtp1442633686-103 Acceptor0 SelectChannelConnector@0.0.0.0:17211, state=RUNNABLE, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=28, name=qtp806943770-28 Acceptor1 SelectChannelConnector@0.0.0.0:19540, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=29, name=qtp806943770-29 Acceptor2 SelectChannelConnector@0.0.0.0:19540, state=RUNNABLE, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([8573C2ED46DC4A5E]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.RecoveryZkTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=27, name=qtp806943770-27 Acceptor0 SelectChannelConnector@0.0.0.0:19540, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=30, name=qtp806943770-30 Acceptor3 SelectChannelConnector@0.0.0.0:19540, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=104, name=qtp1442633686-104 Acceptor1 SelectChannelConnector@0.0.0.0:17211, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=105, name=qtp1442633686-105 Acceptor2 SelectChannelConnector@0.0.0.0:17211, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=106, name=qtp1442633686-106 Acceptor3 SelectChannelConnector@0.0.0.0:17211, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=103, name=qtp1442633686-103 Acceptor0 SelectChannelConnector@0.0.0.0:17211, state=RUNNABLE, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=28, name=qtp806943770-28 Acceptor1 SelectChannelConnector@0.0.0.0:19540, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=29, name=qtp806943770-29 Acceptor2 SelectChannelConnector@0.0.0.0:19540, state=RUNNABLE, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=27, name=qtp806943770-27 Acceptor0 SelectChannelConnector@0.0.0.0:19540, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=30, name=qtp806943770-30 Acceptor3 SelectChannelConnector@0.0.0.0:19540, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=104, name=qtp1442633686-104 Acceptor1 SelectChannelConnector@0.0.0.0:17211, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=105, name=qtp1442633686-105 Acceptor2 SelectChannelConnector@0.0.0.0:17211, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=106, name=qtp1442633686-106 Acceptor3 SelectChannelConnector@0.0.0.0:17211, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=103, name=qtp1442633686-103 Acceptor0 SelectChannelConnector@0.0.0.0:17211, state=RUNNABLE, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=28, name=qtp806943770-28 Acceptor1 SelectChannelConnector@0.0.0.0:19540, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=29, name=qtp806943770-29 Acceptor2 SelectChannelConnector@0.0.0.0:19540, state=RUNNABLE, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([8573C2ED46DC4A5E]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.component.DistributedSpellCheckComponentTest

Error Message:
36 threads leaked from SUITE scope at org.apache.solr.handler.component.DistributedSpellCheckComponentTest:     1) Thread[id=188, name=qtp109855678-188 Acceptor1 SelectChannelConnector@0.0.0.0:19714, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=153, name=qtp1942734199-153 Acceptor1 SelectChannelConnector@0.0.0.0:10507, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=142, name=qtp1432551238-142 Acceptor3 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=174, name=qtp1979690111-174 Acceptor1 SelectChannelConnector@0.0.0.0:15460, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=202, name=qtp406322006-202 Acceptor1 SelectChannelConnector@0.0.0.0:30562, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=155, name=qtp1942734199-155 Acceptor3 SelectChannelConnector@0.0.0.0:10507, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=204, name=qtp406322006-204 Acceptor3 SelectChannelConnector@0.0.0.0:30562, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=215, name=qtp453856425-215 Acceptor0 SelectChannelConnector@0.0.0.0:25011, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=97, name=qtp1431523121-97 Acceptor1 SelectChannelConnector@0.0.0.0:19597, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=99, name=qtp1431523121-99 Acceptor3 SelectChannelConnector@0.0.0.0:19597, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=187, name=qtp109855678-187 Acceptor0 SelectChannelConnector@0.0.0.0:19714, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=216, name=qtp453856425-216 Acceptor1 SelectChannelConnector@0.0.0.0:25011, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=129, name=qtp390636756-129 Acceptor3 SelectChannelConnector@0.0.0.0:51378, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=112, name=qtp1507644908-112 Acceptor2 SelectChannelConnector@0.0.0.0:28779, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=201, name=qtp406322006-201 Acceptor0 SelectChannelConnector@0.0.0.0:30562, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=176, name=qtp1979690111-176 Acceptor3 SelectChannelConnector@0.0.0.0:15460, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=98, name=qtp1431523121-98 Acceptor2 SelectChannelConnector@0.0.0.0:19597, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=175, name=qtp1979690111-175 Acceptor2 SelectChannelConnector@0.0.0.0:15460, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=126, name=qtp390636756-126 Acceptor0 SelectChannelConnector@0.0.0.0:51378, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=113, name=qtp1507644908-113 Acceptor3 SelectChannelConnector@0.0.0.0:28779, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   21) Thread[id=190, name=qtp109855678-190 Acceptor3 SelectChannelConnector@0.0.0.0:19714, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   22) Thread[id=111, name=qtp1507644908-111 Acceptor1 SelectChannelConnector@0.0.0.0:28779, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   23) Thread[id=189, name=qtp109855678-189 Acceptor2 SelectChannelConnector@0.0.0.0:19714, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   24) Thread[id=217, name=qtp453856425-217 Acceptor2 SelectChannelConnector@0.0.0.0:25011, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   25) Thread[id=173, name=qtp1979690111-173 Acceptor0 SelectChannelConnector@0.0.0.0:15460, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   26) Thread[id=152, name=qtp1942734199-152 Acceptor0 SelectChannelConnector@0.0.0.0:10507, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   27) Thread[id=203, name=qtp406322006-203 Acceptor2 SelectChannelConnector@0.0.0.0:30562, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   28) Thread[id=127, name=qtp390636756-127 Acceptor1 SelectChannelConnector@0.0.0.0:51378, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   29) Thread[id=110, name=qtp1507644908-110 Acceptor0 SelectChannelConnector@0.0.0.0:28779, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   30) Thread[id=154, name=qtp1942734199-154 Acceptor2 SelectChannelConnector@0.0.0.0:10507, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   31) Thread[id=96, name=qtp1431523121-96 Acceptor0 SelectChannelConnector@0.0.0.0:19597, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   32) Thread[id=140, name=qtp1432551238-140 Acceptor1 SelectChannelConnector@0.0.0.0:49279, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   33) Thread[id=218, name=qtp453856425-218 Acceptor3 SelectChannelConnector@0.0.0.0:25011, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   34) Thread[id=128, name=qtp390636756-128 Acceptor2 SelectChannelConnector@0.0.0.0:51378, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   35) Thread[id=141, name=qtp1432551238-141 Acceptor2 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   36) Thread[id=139, name=qtp1432551238-139 Acceptor0 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 36 threads leaked from SUITE scope at org.apache.solr.handler.component.DistributedSpellCheckComponentTest: 
   1) Thread[id=188, name=qtp109855678-188 Acceptor1 SelectChannelConnector@0.0.0.0:19714, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=153, name=qtp1942734199-153 Acceptor1 SelectChannelConnector@0.0.0.0:10507, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=142, name=qtp1432551238-142 Acceptor3 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=174, name=qtp1979690111-174 Acceptor1 SelectChannelConnector@0.0.0.0:15460, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=202, name=qtp406322006-202 Acceptor1 SelectChannelConnector@0.0.0.0:30562, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=155, name=qtp1942734199-155 Acceptor3 SelectChannelConnector@0.0.0.0:10507, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=204, name=qtp406322006-204 Acceptor3 SelectChannelConnector@0.0.0.0:30562, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=215, name=qtp453856425-215 Acceptor0 SelectChannelConnector@0.0.0.0:25011, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=97, name=qtp1431523121-97 Acceptor1 SelectChannelConnector@0.0.0.0:19597, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=99, name=qtp1431523121-99 Acceptor3 SelectChannelConnector@0.0.0.0:19597, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=187, name=qtp109855678-187 Acceptor0 SelectChannelConnector@0.0.0.0:19714, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=216, name=qtp453856425-216 Acceptor1 SelectChannelConnector@0.0.0.0:25011, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=129, name=qtp390636756-129 Acceptor3 SelectChannelConnector@0.0.0.0:51378, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=112, name=qtp1507644908-112 Acceptor2 SelectChannelConnector@0.0.0.0:28779, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=201, name=qtp406322006-201 Acceptor0 SelectChannelConnector@0.0.0.0:30562, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=176, name=qtp1979690111-176 Acceptor3 SelectChannelConnector@0.0.0.0:15460, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=98, name=qtp1431523121-98 Acceptor2 SelectChannelConnector@0.0.0.0:19597, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=175, name=qtp1979690111-175 Acceptor2 SelectChannelConnector@0.0.0.0:15460, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=126, name=qtp390636756-126 Acceptor0 SelectChannelConnector@0.0.0.0:51378, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=113, name=qtp1507644908-113 Acceptor3 SelectChannelConnector@0.0.0.0:28779, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  21) Thread[id=190, name=qtp109855678-190 Acceptor3 SelectChannelConnector@0.0.0.0:19714, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  22) Thread[id=111, name=qtp1507644908-111 Acceptor1 SelectChannelConnector@0.0.0.0:28779, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  23) Thread[id=189, name=qtp109855678-189 Acceptor2 SelectChannelConnector@0.0.0.0:19714, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  24) Thread[id=217, name=qtp453856425-217 Acceptor2 SelectChannelConnector@0.0.0.0:25011, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  25) Thread[id=173, name=qtp1979690111-173 Acceptor0 SelectChannelConnector@0.0.0.0:15460, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  26) Thread[id=152, name=qtp1942734199-152 Acceptor0 SelectChannelConnector@0.0.0.0:10507, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  27) Thread[id=203, name=qtp406322006-203 Acceptor2 SelectChannelConnector@0.0.0.0:30562, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  28) Thread[id=127, name=qtp390636756-127 Acceptor1 SelectChannelConnector@0.0.0.0:51378, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  29) Thread[id=110, name=qtp1507644908-110 Acceptor0 SelectChannelConnector@0.0.0.0:28779, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  30) Thread[id=154, name=qtp1942734199-154 Acceptor2 SelectChannelConnector@0.0.0.0:10507, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  31) Thread[id=96, name=qtp1431523121-96 Acceptor0 SelectChannelConnector@0.0.0.0:19597, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  32) Thread[id=140, name=qtp1432551238-140 Acceptor1 SelectChannelConnector@0.0.0.0:49279, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  33) Thread[id=218, name=qtp453856425-218 Acceptor3 SelectChannelConnector@0.0.0.0:25011, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  34) Thread[id=128, name=qtp390636756-128 Acceptor2 SelectChannelConnector@0.0.0.0:51378, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  35) Thread[id=141, name=qtp1432551238-141 Acceptor2 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  36) Thread[id=139, name=qtp1432551238-139 Acceptor0 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([8573C2ED46DC4A5E]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.component.DistributedSpellCheckComponentTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=188, name=qtp109855678-188 Acceptor1 SelectChannelConnector@0.0.0.0:19714, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=153, name=qtp1942734199-153 Acceptor1 SelectChannelConnector@0.0.0.0:10507, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=142, name=qtp1432551238-142 Acceptor3 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=174, name=qtp1979690111-174 Acceptor1 SelectChannelConnector@0.0.0.0:15460, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=202, name=qtp406322006-202 Acceptor1 SelectChannelConnector@0.0.0.0:30562, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=155, name=qtp1942734199-155 Acceptor3 SelectChannelConnector@0.0.0.0:10507, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=204, name=qtp406322006-204 Acceptor3 SelectChannelConnector@0.0.0.0:30562, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=215, name=qtp453856425-215 Acceptor0 SelectChannelConnector@0.0.0.0:25011, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=97, name=qtp1431523121-97 Acceptor1 SelectChannelConnector@0.0.0.0:19597, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=99, name=qtp1431523121-99 Acceptor3 SelectChannelConnector@0.0.0.0:19597, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=187, name=qtp109855678-187 Acceptor0 SelectChannelConnector@0.0.0.0:19714, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=216, name=qtp453856425-216 Acceptor1 SelectChannelConnector@0.0.0.0:25011, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=129, name=qtp390636756-129 Acceptor3 SelectChannelConnector@0.0.0.0:51378, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=112, name=qtp1507644908-112 Acceptor2 SelectChannelConnector@0.0.0.0:28779, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=176, name=qtp1979690111-176 Acceptor3 SelectChannelConnector@0.0.0.0:15460, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=201, name=qtp406322006-201 Acceptor0 SelectChannelConnector@0.0.0.0:30562, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=98, name=qtp1431523121-98 Acceptor2 SelectChannelConnector@0.0.0.0:19597, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=175, name=qtp1979690111-175 Acceptor2 SelectChannelConnector@0.0.0.0:15460, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=113, name=qtp1507644908-113 Acceptor3 SelectChannelConnector@0.0.0.0:28779, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=126, name=qtp390636756-126 Acceptor0 SelectChannelConnector@0.0.0.0:51378, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   21) Thread[id=111, name=qtp1507644908-111 Acceptor1 SelectChannelConnector@0.0.0.0:28779, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   22) Thread[id=190, name=qtp109855678-190 Acceptor3 SelectChannelConnector@0.0.0.0:19714, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   23) Thread[id=189, name=qtp109855678-189 Acceptor2 SelectChannelConnector@0.0.0.0:19714, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   24) Thread[id=217, name=qtp453856425-217 Acceptor2 SelectChannelConnector@0.0.0.0:25011, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   25) Thread[id=173, name=qtp1979690111-173 Acceptor0 SelectChannelConnector@0.0.0.0:15460, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   26) Thread[id=152, name=qtp1942734199-152 Acceptor0 SelectChannelConnector@0.0.0.0:10507, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   27) Thread[id=203, name=qtp406322006-203 Acceptor2 SelectChannelConnector@0.0.0.0:30562, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   28) Thread[id=127, name=qtp390636756-127 Acceptor1 SelectChannelConnector@0.0.0.0:51378, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   29) Thread[id=110, name=qtp1507644908-110 Acceptor0 SelectChannelConnector@0.0.0.0:28779, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   30) Thread[id=154, name=qtp1942734199-154 Acceptor2 SelectChannelConnector@0.0.0.0:10507, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   31) Thread[id=96, name=qtp1431523121-96 Acceptor0 SelectChannelConnector@0.0.0.0:19597, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   32) Thread[id=140, name=qtp1432551238-140 Acceptor1 SelectChannelConnector@0.0.0.0:49279, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   33) Thread[id=218, name=qtp453856425-218 Acceptor3 SelectChannelConnector@0.0.0.0:25011, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   34) Thread[id=128, name=qtp390636756-128 Acceptor2 SelectChannelConnector@0.0.0.0:51378, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   35) Thread[id=141, name=qtp1432551238-141 Acceptor2 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   36) Thread[id=139, name=qtp1432551238-139 Acceptor0 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=188, name=qtp109855678-188 Acceptor1 SelectChannelConnector@0.0.0.0:19714, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=153, name=qtp1942734199-153 Acceptor1 SelectChannelConnector@0.0.0.0:10507, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=142, name=qtp1432551238-142 Acceptor3 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=174, name=qtp1979690111-174 Acceptor1 SelectChannelConnector@0.0.0.0:15460, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=202, name=qtp406322006-202 Acceptor1 SelectChannelConnector@0.0.0.0:30562, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=155, name=qtp1942734199-155 Acceptor3 SelectChannelConnector@0.0.0.0:10507, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=204, name=qtp406322006-204 Acceptor3 SelectChannelConnector@0.0.0.0:30562, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=215, name=qtp453856425-215 Acceptor0 SelectChannelConnector@0.0.0.0:25011, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=97, name=qtp1431523121-97 Acceptor1 SelectChannelConnector@0.0.0.0:19597, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=99, name=qtp1431523121-99 Acceptor3 SelectChannelConnector@0.0.0.0:19597, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=187, name=qtp109855678-187 Acceptor0 SelectChannelConnector@0.0.0.0:19714, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=216, name=qtp453856425-216 Acceptor1 SelectChannelConnector@0.0.0.0:25011, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=129, name=qtp390636756-129 Acceptor3 SelectChannelConnector@0.0.0.0:51378, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=112, name=qtp1507644908-112 Acceptor2 SelectChannelConnector@0.0.0.0:28779, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=176, name=qtp1979690111-176 Acceptor3 SelectChannelConnector@0.0.0.0:15460, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=201, name=qtp406322006-201 Acceptor0 SelectChannelConnector@0.0.0.0:30562, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=98, name=qtp1431523121-98 Acceptor2 SelectChannelConnector@0.0.0.0:19597, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=175, name=qtp1979690111-175 Acceptor2 SelectChannelConnector@0.0.0.0:15460, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=113, name=qtp1507644908-113 Acceptor3 SelectChannelConnector@0.0.0.0:28779, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=126, name=qtp390636756-126 Acceptor0 SelectChannelConnector@0.0.0.0:51378, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  21) Thread[id=111, name=qtp1507644908-111 Acceptor1 SelectChannelConnector@0.0.0.0:28779, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  22) Thread[id=190, name=qtp109855678-190 Acceptor3 SelectChannelConnector@0.0.0.0:19714, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  23) Thread[id=189, name=qtp109855678-189 Acceptor2 SelectChannelConnector@0.0.0.0:19714, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  24) Thread[id=217, name=qtp453856425-217 Acceptor2 SelectChannelConnector@0.0.0.0:25011, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  25) Thread[id=173, name=qtp1979690111-173 Acceptor0 SelectChannelConnector@0.0.0.0:15460, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  26) Thread[id=152, name=qtp1942734199-152 Acceptor0 SelectChannelConnector@0.0.0.0:10507, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  27) Thread[id=203, name=qtp406322006-203 Acceptor2 SelectChannelConnector@0.0.0.0:30562, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  28) Thread[id=127, name=qtp390636756-127 Acceptor1 SelectChannelConnector@0.0.0.0:51378, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  29) Thread[id=110, name=qtp1507644908-110 Acceptor0 SelectChannelConnector@0.0.0.0:28779, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  30) Thread[id=154, name=qtp1942734199-154 Acceptor2 SelectChannelConnector@0.0.0.0:10507, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  31) Thread[id=96, name=qtp1431523121-96 Acceptor0 SelectChannelConnector@0.0.0.0:19597, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  32) Thread[id=140, name=qtp1432551238-140 Acceptor1 SelectChannelConnector@0.0.0.0:49279, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  33) Thread[id=218, name=qtp453856425-218 Acceptor3 SelectChannelConnector@0.0.0.0:25011, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  34) Thread[id=128, name=qtp390636756-128 Acceptor2 SelectChannelConnector@0.0.0.0:51378, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  35) Thread[id=141, name=qtp1432551238-141 Acceptor2 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  36) Thread[id=139, name=qtp1432551238-139 Acceptor0 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([8573C2ED46DC4A5E]:0)




Build Log:
[...truncated 8346 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.RecoveryZkTest
[junit4:junit4]   2> 341 T11 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-RecoveryZkTest-1351959534976
[junit4:junit4]   2> 358 T11 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 461 T12 oaz.Environment.logEnv Server environment:zookeeper.version=3.3.6-1366786, built on 07/29/2012 06:22 GMT
[junit4:junit4]   2> 462 T12 oaz.Environment.logEnv Server environment:host.name=lucene.zones.apache.org
[junit4:junit4]   2> 463 T12 oaz.Environment.logEnv Server environment:java.version=1.7.0_09
[junit4:junit4]   2> 463 T12 oaz.Environment.logEnv Server environment:java.vendor=Oracle Corporation
[junit4:junit4]   2> 464 T12 oaz.Environment.logEnv Server environment:java.home=/usr/local/openjdk7/jre
[junit4:junit4]   2> 465 T12 oaz.Environment.logEnv Server environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/common/lucene-analyzers-common-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/highlighter/lucene-highlighter-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/memory/lucene-memory-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/misc/lucene-misc-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/spatial/lucene-spatial-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/suggest/lucene-suggest-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/grouping/lucene-grouping-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queries/lucene-queries-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queryparser/lucene-queryparser-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/easymock-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/guava-r05.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/log4j-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/zookeeper-3.3.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-continuation-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-deploy-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-http-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-io-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-jmx-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-security-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-server-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-servlet-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-util-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-webapp-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-xml-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-javamail.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jdepend.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.4.jar
[junit4:junit4]   2> 470 T12 oaz.Environment.logEnv Server environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
[junit4:junit4]   2> 470 T12 oaz.Environment.logEnv Server environment:java.io.tmpdir=.
[junit4:junit4]   2> 471 T12 oaz.Environment.logEnv Server environment:java.compiler=<NA>
[junit4:junit4]   2> 472 T12 oaz.Environment.logEnv Server environment:os.name=FreeBSD
[junit4:junit4]   2> 472 T12 oaz.Environment.logEnv Server environment:os.arch=amd64
[junit4:junit4]   2> 473 T12 oaz.Environment.logEnv Server environment:os.version=9.0-RELEASE
[junit4:junit4]   2> 473 T12 oaz.Environment.logEnv Server environment:user.name=hudson
[junit4:junit4]   2> 474 T12 oaz.Environment.logEnv Server environment:user.home=/home/hudson
[junit4:junit4]   2> 474 T12 oaz.Environment.logEnv Server environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 518 T12 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 556 T12 oazs.NIOServerCnxn$Factory.<init> binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 568 T11 oasc.ZkTestServer.run start zk server on port:63015
[junit4:junit4]   2> 641 T12 oazsp.FileTxnSnapLog.save Snapshotting: 0
[junit4:junit4]   2> 655 T11 oaz.Environment.logEnv Client environment:zookeeper.version=3.3.6-1366786, built on 07/29/2012 06:22 GMT
[junit4:junit4]   2> 655 T11 oaz.Environment.logEnv Client environment:host.name=lucene.zones.apache.org
[junit4:junit4]   2> 656 T11 oaz.Environment.logEnv Client environment:java.version=1.7.0_09
[junit4:junit4]   2> 657 T11 oaz.Environment.logEnv Client environment:java.vendor=Oracle Corporation
[junit4:junit4]   2> 657 T11 oaz.Environment.logEnv Client environment:java.home=/usr/local/openjdk7/jre
[junit4:junit4]   2> 658 T11 oaz.Environment.logEnv Client environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/common/lucene-analyzers-common-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/highlighter/lucene-highlighter-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/memory/lucene-memory-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/misc/lucene-misc-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/spatial/lucene-spatial-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/suggest/lucene-suggest-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/grouping/lucene-grouping-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queries/lucene-queries-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queryparser/lucene-queryparser-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/easymock-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/guava-r05.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/log4j-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/zookeeper-3.3.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-continuation-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-deploy-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-http-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-io-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-jmx-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-security-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-server-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-servlet-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-util-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-webapp-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-xml-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-javamail.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jdepend.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.4.jar
[junit4:junit4]   2> 661 T11 oaz.Environment.logEnv Client environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
[junit4:junit4]   2> 661 T11 oaz.Environment.logEnv Client environment:java.io.tmpdir=.
[junit4:junit4]   2> 662 T11 oaz.Environment.logEnv Client environment:java.compiler=<NA>
[junit4:junit4]   2> 663 T11 oaz.Environment.logEnv Client environment:os.name=FreeBSD
[junit4:junit4]   2> 663 T11 oaz.Environment.logEnv Client environment:os.arch=amd64
[junit4:junit4]   2> 664 T11 oaz.Environment.logEnv Client environment:os.version=9.0-RELEASE
[junit4:junit4]   2> 665 T11 oaz.Environment.logEnv Client environment:user.name=hudson
[junit4:junit4]   2> 666 T11 oaz.Environment.logEnv Client environment:user.home=/home/hudson
[junit4:junit4]   2> 666 T11 oaz.Environment.logEnv Client environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 671 T11 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63015 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@790d63f8
[junit4:junit4]   2> 731 T15 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:63015
[junit4:junit4]   2> 731 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 736 T15 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63015, initiating session
[junit4:junit4]   2> 737 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:30574
[junit4:junit4]   2> 760 T13 oazs.NIOServerCnxn.doIO WARNING Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
[junit4:junit4]   2> 761 T13 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:30574 (no session established for client)
[junit4:junit4]   2> 762 T15 oaz.ClientCnxn$SendThread.run Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
[junit4:junit4]   2> 2187 T15 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost/127.0.0.1:63015
[junit4:junit4]   2> 2188 T15 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63015, initiating session
[junit4:junit4]   2> 2189 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:26698
[junit4:junit4]   2> 2191 T13 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:26698
[junit4:junit4]   2> 2208 T16 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 2245 T16 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac7115ecc0000 with negotiated timeout 10000 for client /140.211.11.196:26698
[junit4:junit4]   2> 2246 T15 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:63015, sessionid = 0x13ac7115ecc0000, negotiated timeout = 10000
[junit4:junit4]   2> 2278 T17 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@790d63f8 name:ZooKeeperConnection Watcher:127.0.0.1:63015 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2279 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2285 T11 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 2389 T18 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x13ac7115ecc0000
[junit4:junit4]   2> 2392 T17 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 2391 T11 oaz.ZooKeeper.close Session: 0x13ac7115ecc0000 closed
[junit4:junit4]   2> 2393 T13 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:26698 which had sessionid 0x13ac7115ecc0000
[junit4:junit4]   2> 2393 T11 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63015/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@5702cbd3
[junit4:junit4]   2> 2396 T19 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:63015
[junit4:junit4]   2> 2396 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2397 T19 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63015, initiating session
[junit4:junit4]   2> 2398 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:10099
[junit4:junit4]   2> 2399 T13 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:10099
[junit4:junit4]   2> 2403 T16 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac7115ecc0001 with negotiated timeout 10000 for client /140.211.11.196:10099
[junit4:junit4]   2> 2403 T19 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:63015, sessionid = 0x13ac7115ecc0001, negotiated timeout = 10000
[junit4:junit4]   2> 2404 T20 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5702cbd3 name:ZooKeeperConnection Watcher:127.0.0.1:63015/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2405 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2427 T11 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 2435 T11 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 2441 T11 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 2446 T11 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 2452 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2523 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2540 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 2542 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 2549 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2550 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2572 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 2573 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 2579 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 2580 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 2585 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 2586 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 2591 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 2592 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 2597 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 2599 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 2604 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 2605 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 2610 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 2611 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 2616 T18 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x13ac7115ecc0001
[junit4:junit4]   2> 2618 T20 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 2618 T11 oaz.ZooKeeper.close Session: 0x13ac7115ecc0001 closed
[junit4:junit4]   2> 2619 T13 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:10099 which had sessionid 0x13ac7115ecc0001
[junit4:junit4]   2> 3083 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 3380 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:19540
[junit4:junit4]   2> 3667 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3669 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 3670 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4662 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 4668 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4668 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 4713 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 4719 T11 oasc.CoreContainer.<init> New CoreContainer 1069263533
[junit4:junit4]   2> 4720 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4721 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4829 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4864 T11 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:63015/solr
[junit4:junit4]   2> 4920 T11 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63015/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@7659657a
[junit4:junit4]   2> 4921 T35 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:63015
[junit4:junit4]   2> 4921 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4923 T35 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63015, initiating session
[junit4:junit4]   2> 4924 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:31390
[junit4:junit4]   2> 4926 T13 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:31390
[junit4:junit4]   2> 4929 T16 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac7115ecc0002 with negotiated timeout 5000 for client /140.211.11.196:31390
[junit4:junit4]   2> 4929 T35 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:63015, sessionid = 0x13ac7115ecc0002, negotiated timeout = 5000
[junit4:junit4]   2> 4930 T36 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7659657a name:ZooKeeperConnection Watcher:127.0.0.1:63015/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4930 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4953 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0002 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4966 T11 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 4970 T11 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:19540_solr
[junit4:junit4]   2> 4977 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0002 type:delete cxid:0x7 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:19540_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:19540_solr
[junit4:junit4]   2> 4978 T11 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:19540_solr
[junit4:junit4]   2> 5004 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 5005 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 5005 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 5006 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 5006 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 5007 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 5008 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 5008 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 5009 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 5081 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 5723 T11 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 5762 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0002 type:delete cxid:0x15 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 5764 T11 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 5768 T11 oasc.Overseer.start Overseer (id=88602020105945090-127.0.0.1:19540_solr-n_0000000000) starting
[junit4:junit4]   2> 5769 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0002 type:create cxid:0x1a zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5785 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0002 type:create cxid:0x1b zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5787 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0002 type:create cxid:0x1c zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5793 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0002 type:create cxid:0x1d zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5795 T38 oasc.OverseerCollectionProcessor.run Process current queue of collection messages
[junit4:junit4]   2> 5796 T11 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 5800 T11 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 5824 T37 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 5889 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 5890 T11 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 5891 T11 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 5892 T11 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 5894 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 5896 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 5898 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 6135 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 6395 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 6425 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 6500 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 8633 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 8668 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 8673 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 9058 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9081 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9088 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9097 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9189 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1351959534975/control/data/
[junit4:junit4]   2> 9196 T11 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@12f794ce
[junit4:junit4]   2> 9458 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1351959534975/control/data/index/
[junit4:junit4]   2> 9460 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1351959534975/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 9573 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-1351959534975/control/data/index forceNew:false
[junit4:junit4]   2> 10084 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1351959534975/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@46f0474a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 10085 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 10118 T11 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 11080 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 11082 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 11082 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 11092 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 11232 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 11247 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 11253 T11 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 11290 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 11310 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 11359 T11 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 11361 T11 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 11367 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 11368 T11 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 11369 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 11369 T11 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 11398 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 11399 T11 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11400 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11401 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11403 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11404 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11405 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11406 T11 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11426 T11 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 11445 T11 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 11574 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 11608 T11 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 11617 T11 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 11677 T11 oass.SolrIndexSearcher.<init> Opening Searcher@38219f32 main
[junit4:junit4]   2> 11765 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 11765 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 11767 T11 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 11873 T11 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 12273 T39 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@38219f32 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 12291 T11 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 12293 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0002 type:create cxid:0x74 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 12306 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 12307 T11 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:19540/solr shardId:control_shard
[junit4:junit4]   2> 12324 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 12341 T11 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/control_shard/election
[junit4:junit4]   2> 12359 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0002 type:delete cxid:0x84 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 12366 T11 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 12368 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0002 type:create cxid:0x88 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12368 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12370 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"control_shard",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:19540_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:19540/solr"}
[junit4:junit4]   2> 12386 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0002 type:create cxid:0x8b zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 12426 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (1)
[junit4:junit4]   2> 12873 T11 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 12874 T11 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 12875 T11 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:19540/solr/collection1/
[junit4:junit4]   2> 12876 T11 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 12876 T11 oasc.SyncStrategy.syncToMe http://127.0.0.1:19540/solr/collection1/ has no replicas
[junit4:junit4]   2> 12877 T11 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:19540/solr/collection1/
[junit4:junit4]   2> 12878 T11 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/control_shard
[junit4:junit4]   2> 12885 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0002 type:create cxid:0xa9 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12935 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12949 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (1)
[junit4:junit4]   2> 12998 T11 oasc.ZkController.register We are http://127.0.0.1:19540/solr/collection1/ and leader is http://127.0.0.1:19540/solr/collection1/
[junit4:junit4]   2> 13003 T11 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:19540/solr
[junit4:junit4]   2> 13003 T11 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 13004 T11 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 13009 T11 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13010 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 13011 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 13088 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 13303 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 13306 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:34199
[junit4:junit4]   2> 13308 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 13308 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351959547740
[junit4:junit4]   2> 13309 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351959547740/'
[junit4:junit4]   2> 13360 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 13361 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 13361 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351959547740
[junit4:junit4]   2> 13362 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351959547740/solr.xml
[junit4:junit4]   2> 13363 T11 oasc.CoreContainer.<init> New CoreContainer 1693106741
[junit4:junit4]   2> 13364 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351959547740/'
[junit4:junit4]   2> 13365 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351959547740/'
[junit4:junit4]   2> 13407 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 13429 T11 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:63015/solr
[junit4:junit4]   2> 13430 T11 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63015/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@768db0c6
[junit4:junit4]   2> 13432 T49 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:63015
[junit4:junit4]   2> 13432 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 13434 T49 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63015, initiating session
[junit4:junit4]   2> 13434 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:57041
[junit4:junit4]   2> 13435 T13 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:57041
[junit4:junit4]   2> 13518 T16 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac7115ecc0003 with negotiated timeout 5000 for client /140.211.11.196:57041
[junit4:junit4]   2> 13518 T49 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:63015, sessionid = 0x13ac7115ecc0003, negotiated timeout = 5000
[junit4:junit4]   2> 13520 T50 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@768db0c6 name:ZooKeeperConnection Watcher:127.0.0.1:63015/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 13520 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 13521 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13522 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0003 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13524 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0003 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13525 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"control_shard",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:19540_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:19540/solr"}
[junit4:junit4]   2> 13536 T11 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34199_solr
[junit4:junit4]   2> 13537 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0003 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:34199_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:34199_solr
[junit4:junit4]   2> 13539 T11 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34199_solr
[junit4:junit4]   2> 13542 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (1)
[junit4:junit4]   2> 13546 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 13547 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 13548 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 13548 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 13549 T36 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 13549 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 13550 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 13550 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 13551 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 13551 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 13552 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 13577 T11 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 13585 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351959547740/collection1
[junit4:junit4]   2> 13586 T11 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 13587 T11 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 13588 T11 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 13590 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351959547740/collection1/'
[junit4:junit4]   2> 13592 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-jetty1-1351959547740/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 13593 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-jetty1-1351959547740/collection1/lib/README' to classloader
[junit4:junit4]   2> 13687 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 13774 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 13776 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 13822 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 14489 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 14504 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 14509 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 14537 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 14544 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 14550 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 14554 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 14559 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351959547740/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1351959534975/jetty1/
[junit4:junit4]   2> 14559 T11 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@12f794ce
[junit4:junit4]   2> 14561 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1351959534975/jetty1/index/
[junit4:junit4]   2> 14562 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1351959534975/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 14563 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-1351959534975/jetty1/index forceNew:false
[junit4:junit4]   2> 14571 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1351959534975/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@82d195a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 14572 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 14574 T11 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 14579 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 14580 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 14581 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 14582 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 14583 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 14584 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 14585 T11 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 14587 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 14589 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 14590 T11 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 14592 T11 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 14593 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 14594 T11 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 14595 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 14596 T11 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 14597 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 14599 T11 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14600 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14602 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14603 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14605 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14606 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14609 T11 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14612 T11 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 14617 T11 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 14637 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 14642 T11 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 14644 T11 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 14648 T11 oass.SolrIndexSearcher.<init> Opening Searcher@6b861ad2 main
[junit4:junit4]   2> 14650 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 14650 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 14652 T11 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 14682 T11 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 14853 T55 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6b861ad2 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 14865 T11 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15054 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15055 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:34199_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:34199/solr"}
[junit4:junit4]   2> 15055 T37 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4]   2> 15084 T50 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 15084 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 15389 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 15389 T11 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:34199/solr shardId:shard1
[junit4:junit4]   2> 15391 T11 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 15411 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0003 type:delete cxid:0x42 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr

[...truncated too long message...]

se.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   27) Thread[id=203, name=qtp406322006-203 Acceptor2 SelectChannelConnector@0.0.0.0:30562, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   28) Thread[id=127, name=qtp390636756-127 Acceptor1 SelectChannelConnector@0.0.0.0:51378, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   29) Thread[id=110, name=qtp1507644908-110 Acceptor0 SelectChannelConnector@0.0.0.0:28779, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   30) Thread[id=154, name=qtp1942734199-154 Acceptor2 SelectChannelConnector@0.0.0.0:10507, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   31) Thread[id=96, name=qtp1431523121-96 Acceptor0 SelectChannelConnector@0.0.0.0:19597, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   32) Thread[id=140, name=qtp1432551238-140 Acceptor1 SelectChannelConnector@0.0.0.0:49279, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   33) Thread[id=218, name=qtp453856425-218 Acceptor3 SelectChannelConnector@0.0.0.0:25011, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   34) Thread[id=128, name=qtp390636756-128 Acceptor2 SelectChannelConnector@0.0.0.0:51378, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   35) Thread[id=141, name=qtp1432551238-141 Acceptor2 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   36) Thread[id=139, name=qtp1432551238-139 Acceptor0 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([8573C2ED46DC4A5E]:0)
[junit4:junit4] Completed on J0 in 40.15s, 1 test, 2 errors <<< FAILURES!

[...truncated 473 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:335: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/module-build.xml:61: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:1163: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:827: There were test failures: 233 suites, 965 tests, 4 suite-level errors, 825 ignored

Total time: 31 minutes 10 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure

[JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 609 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.

Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/609/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.LeaderElectionIntegrationTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([74596681D419C368]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.request.TestRemoteStreaming

Error Message:
4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming:     1) Thread[id=35, name=qtp1053008442-35 Acceptor0 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=37, name=qtp1053008442-37 Acceptor2 SelectChannelConnector@0.0.0.0:50599, state=RUNNABLE, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=38, name=qtp1053008442-38 Acceptor3 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=36, name=qtp1053008442-36 Acceptor1 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming: 
   1) Thread[id=35, name=qtp1053008442-35 Acceptor0 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=37, name=qtp1053008442-37 Acceptor2 SelectChannelConnector@0.0.0.0:50599, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=38, name=qtp1053008442-38 Acceptor3 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=36, name=qtp1053008442-36 Acceptor1 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([74596681D419C368]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.request.TestRemoteStreaming

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=35, name=qtp1053008442-35 Acceptor0 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=37, name=qtp1053008442-37 Acceptor2 SelectChannelConnector@0.0.0.0:50599, state=RUNNABLE, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=38, name=qtp1053008442-38 Acceptor3 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=36, name=qtp1053008442-36 Acceptor1 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=35, name=qtp1053008442-35 Acceptor0 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=37, name=qtp1053008442-37 Acceptor2 SelectChannelConnector@0.0.0.0:50599, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=38, name=qtp1053008442-38 Acceptor3 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=36, name=qtp1053008442-36 Acceptor1 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([74596681D419C368]:0)


FAILED:  org.apache.solr.cloud.LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([74596681D419C368]:0)




Build Log:
[...truncated 8297 lines...]
[junit4:junit4] Suite: org.apache.solr.request.TestRemoteStreaming
[junit4:junit4]   2> 0 T28 oas.SolrTestCaseJ4.initCore ####initCore
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-TestRemoteStreaming-1351906047269
[junit4:junit4]   2> 2 T28 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4:junit4]   2> 573 T28 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 917 T28 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:50599
[junit4:junit4]   2> 959 T28 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 960 T28 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: solr/
[junit4:junit4]   2> 960 T28 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: 'solr/'
[junit4:junit4]   2> 1004 T28 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1005 T28 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1005 T28 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: solr/
[junit4:junit4]   2> 1006 T28 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solr/solr.xml
[junit4:junit4]   2> 1006 T28 oasc.CoreContainer.<init> New CoreContainer 1645340219
[junit4:junit4]   2> 1007 T28 oasc.CoreContainer$Initializer.initialize no solr.xml file found - using default
[junit4:junit4]   2> 1008 T28 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'solr/'
[junit4:junit4]   2> 1008 T28 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'solr/'
[junit4:junit4]   2> 1053 T28 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 1109 T28 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: solr/collection1
[junit4:junit4]   2> 1110 T28 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'solr/collection1/'
[junit4:junit4]   2> 1186 T28 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 1257 T28 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1261 T28 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 1290 T28 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1883 T28 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1896 T28 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1900 T28 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1943 T28 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1967 T28 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1974 T28 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1982 T28 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1986 T28 oasc.SolrCore.<init> [collection1] Opening new SolrCore at solr/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-TestRemoteStreaming-1351906047269/
[junit4:junit4]   2> 1986 T28 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2bf96987
[junit4:junit4]   2> 1989 T28 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-TestRemoteStreaming-1351906047269/index/
[junit4:junit4]   2> 1990 T28 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-TestRemoteStreaming-1351906047269/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1992 T28 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-TestRemoteStreaming-1351906047269/index forceNew:false
[junit4:junit4]   2> 1996 T28 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4bc2ebc7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3c3695),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1997 T28 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 1999 T28 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 2004 T28 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2006 T28 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 2007 T28 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2008 T28 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 2010 T28 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2011 T28 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2012 T28 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2014 T28 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2015 T28 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2017 T28 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 2019 T28 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 2020 T28 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2021 T28 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 2022 T28 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 2024 T28 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 2025 T28 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2027 T28 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2028 T28 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2030 T28 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2032 T28 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2033 T28 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2035 T28 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2036 T28 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2038 T28 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 2039 T28 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 2063 T28 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2072 T28 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 2075 T28 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: []
[junit4:junit4]   2> 2080 T28 oass.SolrIndexSearcher.<init> Opening Searcher@33a40908 main
[junit4:junit4]   2> 2081 T28 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 2083 T28 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2084 T28 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2085 T28 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 2129 T28 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 2361 T28 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 2362 T28 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 2363 T28 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 2364 T28 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 2364 T28 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 2365 T28 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 2365 T28 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 2366 T28 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 2367 T28 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 2367 T28 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 2391 T43 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@33a40908 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2405 T28 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 2406 T28 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 2406 T28 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 2407 T28 oas.SolrJettyTestBase.createJetty Jetty Assigned Port#50599
[junit4:junit4]   2> 2424 T28 oas.SolrTestCaseJ4.setUp ###Starting testNoUrlAccess
[junit4:junit4]   2> 2526 T28 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C5 name=collection1 org.apache.solr.core.SolrCore@4d8267cc
[junit4:junit4]   2> 3774 T40 C5 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4bc2ebc7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3c3695),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 3774 T40 C5 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 3776 T40 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1234]} 0 33
[junit4:junit4]   2> 3910 T41 C5 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 3948 T41 C5 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4bc2ebc7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3c3695),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4bc2ebc7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3c3695),segFN=segments_2,generation=2,filenames=[_0.fnm, _0_Memory_0.ram, _0_MockVariableIntBlock_0.doc, _0_nrm.cfs, _0_MockVariableIntBlock_0.skp, _0_Memory_1.ram, _0_MockVariableIntBlock_0.pyl, _0_Pulsing41_0.doc, _0_Pulsing41_0.pos, _0_MockVariableIntBlock_0.tib, _0_Pulsing41_0.tim, _0_Pulsing41_0.tip, _0_MockVariableIntBlock_0.frq, _0_nrm.cfe, _0_MockVariableIntBlock_0.pos, segments_2, _0.fdx, _0.si, _0.fdt, _0_MockVariableIntBlock_0.tii]
[junit4:junit4]   2> 3948 T41 C5 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 3966 T41 C5 oass.SolrIndexSearcher.<init> Opening Searcher@4f8771a main
[junit4:junit4]   2> 3966 T41 C5 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 3967 T41 C5 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 3969 T43 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4f8771a main{StandardDirectoryReader(segments_2:3 _0(4.1):C1)}
[junit4:junit4]   2> 3970 T41 C5 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 60
[junit4:junit4]   2> 4051 T42 C5 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=56 
[junit4:junit4]   2> 4062 T41 C5 REQ [collection1] webapp=/solr path=/select params={wt=javabin&stream.url=http://127.0.0.1:50599/solr/update?commit%3Dtrue%26stream.body%3D%253Cdelete%253E%253Cquery%253E*%253A*%253C%252Fquery%253E%253C%252Fdelete%253E&q=*:*&version=2} hits=1 status=0 QTime=5 
[junit4:junit4]   2> 4066 T42 C5 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 4067 T28 oas.SolrTestCaseJ4.tearDown ###Ending testNoUrlAccess
[junit4:junit4]   2> 4076 T28 oas.SolrTestCaseJ4.setUp ###Starting testStreamUrl
[junit4:junit4]   2> 4080 T40 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1234]} 0 1
[junit4:junit4]   2> 4084 T41 C5 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 4089 T41 C5 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4bc2ebc7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3c3695),segFN=segments_2,generation=2,filenames=[_0.fnm, _0_Memory_0.ram, _0_MockVariableIntBlock_0.doc, _0_nrm.cfs, _0_MockVariableIntBlock_0.skp, _0_Memory_1.ram, _0_MockVariableIntBlock_0.pyl, _0_Pulsing41_0.doc, _0_Pulsing41_0.pos, _0_MockVariableIntBlock_0.tib, _0_Pulsing41_0.tim, _0_Pulsing41_0.tip, _0_MockVariableIntBlock_0.frq, _0_nrm.cfe, _0_MockVariableIntBlock_0.pos, segments_2, _0.fdx, _0.si, _0.fdt, _0_MockVariableIntBlock_0.tii]
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4bc2ebc7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3c3695),segFN=segments_3,generation=3,filenames=[_1_MockVariableIntBlock_0.skp, _1_nrm.cfs, _1_Memory_1.ram, _1_MockVariableIntBlock_0.tii, _1_MockVariableIntBlock_0.pos, _1_MockVariableIntBlock_0.pyl, _1_Pulsing41_0.doc, _1.fnm, _1_Pulsing41_0.pos, _1_nrm.cfe, _1_MockVariableIntBlock_0.tib, _1_MockVariableIntBlock_0.frq, _1.fdx, _1_Memory_0.ram, _1_Pulsing41_0.tip, _1.fdt, _1_Pulsing41_0.tim, _1.si, _1_MockVariableIntBlock_0.doc, segments_3]
[junit4:junit4]   2> 4090 T41 C5 oasc.SolrDeletionPolicy.updateCommits newest commit = 3
[junit4:junit4]   2> 4092 T41 C5 oass.SolrIndexSearcher.<init> Opening Searcher@11ad05de main
[junit4:junit4]   2> 4093 T41 C5 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 4093 T41 C5 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 4096 T43 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@11ad05de main{StandardDirectoryReader(segments_3:7 _1(4.1):C1)}
[junit4:junit4]   2> 4097 T41 C5 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 13
[junit4:junit4]   2> 4105 T42 C5 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 4121 T42 C5 REQ [collection1] webapp=/solr path=/select params={wt=csv&q=*:*&fl=id} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 4148 T41 C5 REQ [collection1] webapp=/solr path=/debug/dump params={wt=xml&stream.url=http://127.0.0.1:50599/solr/select?q%3D*:*%26fl%3Did%26wt%3Dcsv} status=0 QTime=30 
[junit4:junit4]   2> 4157 T28 oas.SolrTestCaseJ4.tearDown ###Ending testStreamUrl
[junit4:junit4]   2> 4166 T28 oas.SolrTestCaseJ4.setUp ###Starting testMakeDeleteAllUrl
[junit4:junit4]   2> 4171 T40 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1234]} 0 1
[junit4:junit4]   2> 4175 T41 C5 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 4180 T41 C5 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4bc2ebc7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3c3695),segFN=segments_3,generation=3,filenames=[_1_MockVariableIntBlock_0.skp, _1_nrm.cfs, _1_Memory_1.ram, _1_MockVariableIntBlock_0.tii, _1_MockVariableIntBlock_0.pos, _1_MockVariableIntBlock_0.pyl, _1_Pulsing41_0.doc, _1.fnm, _1_Pulsing41_0.pos, _1_nrm.cfe, _1_MockVariableIntBlock_0.tib, _1_MockVariableIntBlock_0.frq, _1.fdx, _1_Memory_0.ram, _1_Pulsing41_0.tip, _1.fdt, _1_Pulsing41_0.tim, _1.si, _1_MockVariableIntBlock_0.doc, segments_3]
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4bc2ebc7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3c3695),segFN=segments_4,generation=4,filenames=[_2_Pulsing41_0.doc, _2.si, _2_MockVariableIntBlock_0.tii, _2_MockVariableIntBlock_0.pyl, _2_MockVariableIntBlock_0.tib, _2.fdt, _2_MockVariableIntBlock_0.frq, _2.fdx, _2_MockVariableIntBlock_0.doc, _2.fnm, _2_Pulsing41_0.tim, _2_Pulsing41_0.pos, _2_nrm.cfe, _2_Memory_1.ram, _2_Pulsing41_0.tip, _2_Memory_0.ram, _2_MockVariableIntBlock_0.skp, _2_nrm.cfs, _2_MockVariableIntBlock_0.pos, segments_4]
[junit4:junit4]   2> 4181 T41 C5 oasc.SolrDeletionPolicy.updateCommits newest commit = 4
[junit4:junit4]   2> 4183 T41 C5 oass.SolrIndexSearcher.<init> Opening Searcher@24fd5136 main
[junit4:junit4]   2> 4184 T41 C5 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 4184 T41 C5 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 4187 T43 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@24fd5136 main{StandardDirectoryReader(segments_4:11 _2(4.1):C1)}
[junit4:junit4]   2> 4188 T41 C5 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 13
[junit4:junit4]   2> 4192 T42 C5 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 4195 T40 C5 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4:junit4]   2> 4196 T40 C5 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 4197 T40 C5 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4bc2ebc7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3c3695),segFN=segments_4,generation=4,filenames=[_2_Pulsing41_0.doc, _2.si, _2_MockVariableIntBlock_0.tii, _2_MockVariableIntBlock_0.pyl, _2_MockVariableIntBlock_0.tib, _2.fdt, _2_MockVariableIntBlock_0.frq, _2.fdx, _2_MockVariableIntBlock_0.doc, _2.fnm, _2_Pulsing41_0.tim, _2_Pulsing41_0.pos, _2_nrm.cfe, _2_Memory_1.ram, _2_Pulsing41_0.tip, _2_Memory_0.ram, _2_MockVariableIntBlock_0.skp, _2_nrm.cfs, _2_MockVariableIntBlock_0.pos, segments_4]
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4bc2ebc7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3c3695),segFN=segments_5,generation=5,filenames=[segments_5]
[junit4:junit4]   2> 4197 T40 C5 oasc.SolrDeletionPolicy.updateCommits newest commit = 5
[junit4:junit4]   2> 4198 T40 C5 oass.SolrIndexSearcher.<init> Opening Searcher@782a437c main
[junit4:junit4]   2> 4199 T40 C5 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 4199 T40 C5 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 4201 T43 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@782a437c main{StandardDirectoryReader(segments_5:12)}
[junit4:junit4]   2> 4202 T40 C5 UPDATE [collection1] webapp=/solr path=/update params={commit=true&stream.body=<delete><query>*:*</query></delete>} {deleteByQuery=*:*,commit=} 0 8
[junit4:junit4]   2> 4207 T41 C5 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 4208 T28 oas.SolrTestCaseJ4.tearDown ###Ending testMakeDeleteAllUrl
[junit4:junit4]   2> 4216 T28 oas.SolrTestCaseJ4.setUp ###Starting testQtUpdateFails
[junit4:junit4]   2> 4222 T42 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1234]} 0 2
[junit4:junit4]   2> 4227 T40 C5 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 4233 T40 C5 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4bc2ebc7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3c3695),segFN=segments_5,generation=5,filenames=[segments_5]
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4bc2ebc7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3c3695),segFN=segments_6,generation=6,filenames=[_3_nrm.cfe, _3_MockVariableIntBlock_0.frq, _3_MockVariableIntBlock_0.doc, _3.fnm, _3.fdt, _3_MockVariableIntBlock_0.tii, _3.fdx, _3_Pulsing41_0.pos, _3_nrm.cfs, _3_MockVariableIntBlock_0.pyl, _3_MockVariableIntBlock_0.tib, _3_Pulsing41_0.tim, _3.si, _3_Pulsing41_0.tip, _3_Memory_0.ram, _3_MockVariableIntBlock_0.skp, segments_6, _3_Pulsing41_0.doc, _3_Memory_1.ram, _3_MockVariableIntBlock_0.pos]
[junit4:junit4]   2> 4233 T40 C5 oasc.SolrDeletionPolicy.updateCommits newest commit = 6
[junit4:junit4]   2> 4237 T40 C5 oass.SolrIndexSearcher.<init> Opening Searcher@74e6cbcd main
[junit4:junit4]   2> 4237 T40 C5 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 4238 T40 C5 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 4241 T43 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@74e6cbcd main{StandardDirectoryReader(segments_6:14 _3(4.1):C1)}
[junit4:junit4]   2> 4241 T40 C5 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 14
[junit4:junit4]   2> 4246 T41 C5 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 4250 T28 oas.SolrTestCaseJ4.tearDown ###Ending testQtUpdateFails
[junit4:junit4]   2> 4298 T28 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1645340219
[junit4:junit4]   2> 4298 T28 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@4d8267cc
[junit4:junit4]   2> 4313 T28 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=5,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=4,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4]   2> 4314 T28 oasc.SolrCore.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 4314 T28 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 4315 T28 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 4405 T28 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 4417 T28 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 4518 T28 oejut.QueuedThreadPool.doStop WARNING 4 threads could not be stopped
[junit4:junit4]   2> 4518 T28 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> 4532 T27 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 4 leaked thread(s).
[junit4:junit4]   2> 24664 T27 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming: 
[junit4:junit4]   2> 	   1) Thread[id=35, name=qtp1053008442-35 Acceptor0 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   2) Thread[id=37, name=qtp1053008442-37 Acceptor2 SelectChannelConnector@0.0.0.0:50599, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   3) Thread[id=38, name=qtp1053008442-38 Acceptor3 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   4) Thread[id=36, name=qtp1053008442-36 Acceptor1 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 24665 T27 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
[junit4:junit4]   2> 	   1) Thread[id=35, name=qtp1053008442-35 Acceptor0 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	   2) Thread[id=37, name=qtp1053008442-37 Acceptor2 SelectChannelConnector@0.0.0.0:50599, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	   3) Thread[id=38, name=qtp1053008442-38 Acceptor3 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	   4) Thread[id=36, name=qtp1053008442-36 Acceptor1 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 27708 T27 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
[junit4:junit4]   2> 	   1) Thread[id=35, name=qtp1053008442-35 Acceptor0 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   2) Thread[id=37, name=qtp1053008442-37 Acceptor2 SelectChannelConnector@0.0.0.0:50599, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   3) Thread[id=38, name=qtp1053008442-38 Acceptor3 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   4) Thread[id=36, name=qtp1053008442-36 Acceptor1 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene41: {timestamp=Pulsing41(freqCutoff=16 minBlockSize=6 maxBlockSize=102), range_facet_l=Pulsing41(freqCutoff=16 minBlockSize=6 maxBlockSize=102), id=PostingsFormat(name=Memory doPackFST= false), range_facet_sl=PostingsFormat(name=Memory doPackFST= true), range_facet_si=MockVariableIntBlock(baseBlockSize=86), multiDefault=MockVariableIntBlock(baseBlockSize=86), intDefault=Pulsing41(freqCutoff=16 minBlockSize=6 maxBlockSize=102)}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=fi, timezone=Africa/Algiers
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=5,free=38914040,total=145752064
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestCharFilters, SoftAutoCommitTest, TermVectorComponentTest, TestRemoteStreaming]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestRemoteStreaming -Dtests.seed=74596681D419C368 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=fi -Dtests.timezone=Africa/Algiers -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] ERROR   0.00s J1 | TestRemoteStreaming (suite) <<<
[junit4:junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming: 
[junit4:junit4]    >    1) Thread[id=35, name=qtp1053008442-35 Acceptor0 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=37, name=qtp1053008442-37 Acceptor2 SelectChannelConnector@0.0.0.0:50599, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=38, name=qtp1053008442-38 Acceptor3 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=36, name=qtp1053008442-36 Acceptor1 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([74596681D419C368]:0)
[junit4:junit4]    > Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4]    >    1) Thread[id=35, name=qtp1053008442-35 Acceptor0 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=37, name=qtp1053008442-37 Acceptor2 SelectChannelConnector@0.0.0.0:50599, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=38, name=qtp1053008442-38 Acceptor3 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=36, name=qtp1053008442-36 Acceptor1 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([74596681D419C368]:0)
[junit4:junit4] Completed on J1 in 27.92s, 4 tests, 2 errors <<< FAILURES!

[...truncated 639 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.LeaderElectionIntegrationTest
[junit4:junit4]   2> 4 T75 oas.SolrTestCaseJ4.setUp ###Starting testLeaderElectionAfterClientTimeout
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841
[junit4:junit4]   2> 10 T75 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 133 T76 oaz.Environment.logEnv Server environment:zookeeper.version=3.3.6-1366786, built on 07/29/2012 06:22 GMT
[junit4:junit4]   2> 134 T76 oaz.Environment.logEnv Server environment:host.name=lucene.zones.apache.org
[junit4:junit4]   2> 134 T76 oaz.Environment.logEnv Server environment:java.version=1.7.0_09
[junit4:junit4]   2> 134 T76 oaz.Environment.logEnv Server environment:java.vendor=Oracle Corporation
[junit4:junit4]   2> 134 T76 oaz.Environment.logEnv Server environment:java.home=/usr/local/openjdk7/jre
[junit4:junit4]   2> 134 T76 oaz.Environment.logEnv Server environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/common/lucene-analyzers-common-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/highlighter/lucene-highlighter-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/memory/lucene-memory-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/misc/lucene-misc-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/spatial/lucene-spatial-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/suggest/lucene-suggest-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/grouping/lucene-grouping-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queries/lucene-queries-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queryparser/lucene-queryparser-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/easymock-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/guava-r05.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/log4j-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/zookeeper-3.3.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-continuation-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-deploy-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-http-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-io-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-jmx-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-security-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-server-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-servlet-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-util-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-webapp-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-xml-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-javamail.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jdepend.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.4.jar
[junit4:junit4]   2> 135 T76 oaz.Environment.logEnv Server environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
[junit4:junit4]   2> 136 T76 oaz.Environment.logEnv Server environment:java.io.tmpdir=.
[junit4:junit4]   2> 136 T76 oaz.Environment.logEnv Server environment:java.compiler=<NA>
[junit4:junit4]   2> 136 T76 oaz.Environment.logEnv Server environment:os.name=FreeBSD
[junit4:junit4]   2> 136 T76 oaz.Environment.logEnv Server environment:os.arch=amd64
[junit4:junit4]   2> 136 T76 oaz.Environment.logEnv Server environment:os.version=9.0-RELEASE
[junit4:junit4]   2> 137 T76 oaz.Environment.logEnv Server environment:user.name=hudson
[junit4:junit4]   2> 137 T76 oaz.Environment.logEnv Server environment:user.home=/home/hudson
[junit4:junit4]   2> 137 T76 oaz.Environment.logEnv Server environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 189 T76 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 233 T76 oazs.NIOServerCnxn$Factory.<init> binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 315 T76 oazsp.FileTxnSnapLog.save Snapshotting: 0
[junit4:junit4]   2> 318 T75 oasc.ZkTestServer.run start zk server on port:57157
[junit4:junit4]   2> 405 T75 oaz.Environment.logEnv Client environment:zookeeper.version=3.3.6-1366786, built on 07/29/2012 06:22 GMT
[junit4:junit4]   2> 406 T75 oaz.Environment.logEnv Client environment:host.name=lucene.zones.apache.org
[junit4:junit4]   2> 406 T75 oaz.Environment.logEnv Client environment:java.version=1.7.0_09
[junit4:junit4]   2> 406 T75 oaz.Environment.logEnv Client environment:java.vendor=Oracle Corporation
[junit4:junit4]   2> 406 T75 oaz.Environment.logEnv Client environment:java.home=/usr/local/openjdk7/jre
[junit4:junit4]   2> 407 T75 oaz.Environment.logEnv Client environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/common/lucene-analyzers-common-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/highlighter/lucene-highlighter-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/memory/lucene-memory-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/misc/lucene-misc-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/spatial/lucene-spatial-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/suggest/lucene-suggest-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/grouping/lucene-grouping-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queries/lucene-queries-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queryparser/lucene-queryparser-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/easymock-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/guava-r05.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/log4j-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/zookeeper-3.3.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-continuation-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-deploy-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-http-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-io-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-jmx-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-security-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-server-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-servlet-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-util-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-webapp-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-xml-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-javamail.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jdepend.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.4.jar
[junit4:junit4]   2> 408 T75 oaz.Environment.logEnv Client environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
[junit4:junit4]   2> 408 T75 oaz.Environment.logEnv Client environment:java.io.tmpdir=.
[junit4:junit4]   2> 408 T75 oaz.Environment.logEnv Client environment:java.compiler=<NA>
[junit4:junit4]   2> 408 T75 oaz.Environment.logEnv Client environment:os.name=FreeBSD
[junit4:junit4]   2> 409 T75 oaz.Environment.logEnv Client environment:os.arch=amd64
[junit4:junit4]   2> 409 T75 oaz.Environment.logEnv Client environment:os.version=9.0-RELEASE
[junit4:junit4]   2> 409 T75 oaz.Environment.logEnv Client environment:user.name=hudson
[junit4:junit4]   2> 409 T75 oaz.Environment.logEnv Client environment:user.home=/home/hudson
[junit4:junit4]   2> 410 T75 oaz.Environment.logEnv Client environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 415 T75 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57157 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@1bcc1503
[junit4:junit4]   2> 469 T80 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:57157
[junit4:junit4]   2> 470 T75 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 472 T80 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57157, initiating session
[junit4:junit4]   2> 472 T77 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:21567
[junit4:junit4]   2> 485 T77 oazs.NIOServerCnxn.doIO WARNING Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
[junit4:junit4]   2> 485 T77 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:21567 (no session established for client)
[junit4:junit4]   2> 486 T80 oaz.ClientCnxn$SendThread.run Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
[junit4:junit4]   2> 2204 T80 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost/127.0.0.1:57157
[junit4:junit4]   2> 2204 T80 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57157, initiating session
[junit4:junit4]   2> 2205 T77 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:21636
[junit4:junit4]   2> 2206 T77 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:21636
[junit4:junit4]   2> 2222 T79 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 2264 T79 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac3e1b9bc0000 with negotiated timeout 10000 for client /140.211.11.196:21636
[junit4:junit4]   2> 2265 T80 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:57157, sessionid = 0x13ac3e1b9bc0000, negotiated timeout = 10000
[junit4:junit4]   2> 2269 T82 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1bcc1503 name:ZooKeeperConnection Watcher:127.0.0.1:57157 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2270 T75 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2274 T75 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 2349 T81 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x13ac3e1b9bc0000
[junit4:junit4]   2> 2351 T82 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 2351 T75 oaz.ZooKeeper.close Session: 0x13ac3e1b9bc0000 closed
[junit4:junit4]   2> 2351 T75 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57157/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@7b0c2fa
[junit4:junit4]   2> 2352 T77 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:21636 which had sessionid 0x13ac3e1b9bc0000
[junit4:junit4]   2> 2352 T83 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:57157
[junit4:junit4]   2> 2352 T75 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2353 T83 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57157, initiating session
[junit4:junit4]   2> 2354 T77 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:43000
[junit4:junit4]   2> 2354 T77 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:43000
[junit4:junit4]   2> 2356 T79 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac3e1b9bc0001 with negotiated timeout 10000 for client /140.211.11.196:43000
[junit4:junit4]   2> 2356 T83 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:57157, sessionid = 0x13ac3e1b9bc0001, negotiated timeout = 10000
[junit4:junit4]   2> 2356 T84 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7b0c2fa name:ZooKeeperConnection Watcher:127.0.0.1:57157/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2357 T75 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2370 T75 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 2376 T75 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 2380 T75 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 2388 T75 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 2393 T75 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2423 T75 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2438 T75 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 2439 T75 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 2445 T75 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2445 T75 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2465 T75 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 2466 T75 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 2471 T75 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 2472 T75 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 2476 T75 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 2477 T75 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 2481 T75 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 2482 T75 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 2487 T75 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 2487 T75 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 2495 T75 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 2496 T75 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 2500 T75 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 2501 T75 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 2506 T81 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x13ac3e1b9bc0001
[junit4:junit4]   2> 2507 T84 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 2507 T75 oaz.ZooKeeper.close Session: 0x13ac3e1b9bc0001 closed
[junit4:junit4]   2> 2507 T77 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:43000 which had sessionid 0x13ac3e1b9bc0001
[junit4:junit4]   2> 2508 T75 oasc.LeaderElectionIntegrationTest.setUp ####SETUP_START testLeaderElectionAfterClientTimeout
[junit4:junit4]   2> 2510 T75 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2511 T75 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 2511 T75 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 2512 T75 oasc.CoreContainer.<init> New CoreContainer 1755608529
[junit4:junit4]   2> 2512 T75 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2513 T75 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2590 T75 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 2607 T75 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:57157/solr
[junit4:junit4]   2> 2635 T75 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57157/solr sessionTimeout=3000 watcher=org.apache.solr.common.cloud.ConnectionManager@5e934dd
[junit4:junit4]   2> 2637 T85 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:57157
[junit4:junit4]   2> 2637 T75 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2638 T85 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57157, initiating session
[junit4:junit4]   2> 2638 T77 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:10260
[junit4:junit4]   2> 2639 T77 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:10260
[junit4:junit4]   2> 2640 T79 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac3e1b9bc0002 with negotiated timeout 3000 for client /140.211.11.196:10260
[junit4:junit4]   2> 2640 T85 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:57157, sessionid = 0x13ac3e1b9bc0002, negotiated timeout = 3000
[junit4:junit4]   2> 2641 T86 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5e934dd name:ZooKeeperConnection Watcher:127.0.0.1:57157/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2641 T75 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2663 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0002 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2675 T75 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 2678 T75 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:7000_solr
[junit4:junit4]   2> 2684 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0002 type:delete cxid:0x7 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:7000_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:7000_solr
[junit4:junit4]   2> 2685 T75 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:7000_solr
[junit4:junit4]   2> 2690 T75 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 2690 T75 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 2690 T75 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 2691 T75 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 2691 T75 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 2691 T75 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 2692 T75 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 2692 T75 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 2693 T75 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 2693 T75 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 2711 T75 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 2734 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0002 type:delete cxid:0x15 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 2735 T75 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 2739 T75 oasc.Overseer.start Overseer (id=88598516938309634-127.0.0.1:7000_solr-n_0000000000) starting
[junit4:junit4]   2> 2740 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0002 type:create cxid:0x1a zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2766 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0002 type:create cxid:0x1b zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2767 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0002 type:create cxid:0x1c zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2783 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0002 type:create cxid:0x1d zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2784 T88 oasc.OverseerCollectionProcessor.run Process current queue of collection messages
[junit4:junit4]   2> 2785 T75 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 2789 T75 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 2824 T87 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 2868 T75 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 2869 T75 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 2870 T75 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 2870 T75 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 2872 T75 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 2874 T75 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 2875 T75 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 2988 T75 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 3075 T75 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 3077 T75 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 3110 T75 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 3855 T75 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 3870 T75 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 3872 T75 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 3907 T75 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 3918 T75 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 3929 T75 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 3934 T75 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 3936 T75 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7000/
[junit4:junit4]   2> 3939 T75 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@54db16d7
[junit4:junit4]   2> 3939 T75 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7000/index/
[junit4:junit4]   2> 3940 T75 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7000/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 3940 T75 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7000/index forceNew:false
[junit4:junit4]   2> 3942 T75 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2eab8f37 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7c11d6fe),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 3942 T75 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 3943 T75 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 3945 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 3945 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 3946 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 3946 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 3947 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 3947 T75 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 3948 T75 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 3948 T75 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 3948 T75 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 3949 T75 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 3949 T75 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 3950 T75 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 3950 T75 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 3950 T75 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 3951 T75 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 3951 T75 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 3952 T75 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3952 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3952 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3953 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3953 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3954 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3954 T75 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3954 T75 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 3955 T75 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 3963 T75 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 3967 T75 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 3968 T75 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 3970 T75 oass.SolrIndexSearcher.<init> Opening Searcher@75fd56dc main
[junit4:junit4]   2> 3970 T75 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3971 T75 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 3971 T75 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 3971 T75 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 3977 T75 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 4029 T89 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@75fd56dc main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 4033 T75 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 4034 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0002 type:create cxid:0x56 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 4045 T75 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 4045 T75 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:7000/solr shardId:shard1
[junit4:junit4]   2> 4055 T75 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 4066 T75 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 4088 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0002 type:delete cxid:0x65 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 4089 T75 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 4090 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0002 type:create cxid:0x66 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4334 T87 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4335 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7000_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7000/solr"}
[junit4:junit4]   2> 4348 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0002 type:create cxid:0x6d zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 4384 T86 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (1)
[junit4:junit4]   2> 4602 T75 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 4603 T75 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 4603 T75 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:7000/solr/collection1/
[junit4:junit4]   2> 4603 T75 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 4609 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0002 type:create cxid:0x8b zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4892 T87 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4906 T86 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (1)
[junit4:junit4]   2> 4925 T75 oasc.ZkController.register We are http://127.0.0.1:7000/solr/collection1/ and leader is http://127.0.0.1:7000/solr/collection1/
[junit4:junit4]   2> 4925 T75 oasc.ZkController.checkRecovery WARNING Skipping recovery according to sys prop solrcloud.skip.autorecovery
[junit4:junit4]   2> 4926 T75 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 4929 T75 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4933 T75 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4933 T75 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 4933 T75 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 4934 T75 oasc.CoreContainer.<init> New CoreContainer 229421624
[junit4:junit4]   2> 4935 T75 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4935 T75 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4999 T75 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 5016 T75 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:57157/solr
[junit4:junit4]   2> 5017 T75 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57157/solr sessionTimeout=3000 watcher=org.apache.solr.common.cloud.ConnectionManager@577670ee
[junit4:junit4]   2> 5018 T90 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:57157
[junit4:junit4]   2> 5018 T75 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 5019 T90 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57157, initiating session
[junit4:junit4]   2> 5019 T77 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:54083
[junit4:junit4]   2> 5020 T77 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:54083
[junit4:junit4]   2> 5021 T79 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac3e1b9bc0003 with negotiated timeout 3000 for client /140.211.11.196:54083
[junit4:junit4]   2> 5021 T90 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:57157, sessionid = 0x13ac3e1b9bc0003, negotiated timeout = 3000
[junit4:junit4]   2> 5022 T91 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@577670ee name:ZooKeeperConnection Watcher:127.0.0.1:57157/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 5022 T75 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 5023 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0003 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5024 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0003 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5026 T75 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:7001_solr
[junit4:junit4]   2> 5027 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0003 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:7001_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:7001_solr
[junit4:junit4]   2> 5034 T75 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:7001_solr
[junit4:junit4]   2> 5037 T86 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 5038 T75 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 5038 T75 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 5039 T75 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 5039 T75 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 5039 T75 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 5040 T75 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 5040 T75 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 5040 T75 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 5041 T75 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 5041 T75 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 5061 T75 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 5069 T75 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 5069 T75 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 5070 T75 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 5070 T75 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 5072 T75 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 5073 T75 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 5074 T75 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 5140 T75 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 5222 T75 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 5224 T75 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 5254 T75 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 5412 T87 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5413 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7000_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7000/solr"}
[junit4:junit4]   2> 5427 T86 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 5427 T91 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 5887 T75 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 5903 T75 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 5907 T75 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5933 T75 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5940 T75 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5949 T75 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5950 T75 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5952 T75 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7001/
[junit4:junit4]   2> 5953 T75 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@54db16d7
[junit4:junit4]   2> 5954 T75 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7001/index/
[junit4:junit4]   2> 5955 T75 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7001/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5956 T75 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7001/index forceNew:false
[junit4:junit4]   2> 5958 T75 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@66d9ce6e lockFactory=org.apache.lucene.store.NativeFSLockFactory@245b315a; maxCacheMB=0.923828125 maxMergeSizeMB=0.8994140625)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5959 T75 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 5960 T75 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 5962 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 5963 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 5963 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 5964 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 5965 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 5965 T75 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 5966 T75 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 5966 T75 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 5967 T75 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 5968 T75 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 5968 T75 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 5969 T75 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 5970 T75 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 5970 T75 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 5971 T75 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 5971 T75 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 5972 T75 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5973 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5973 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5974 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5975 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5975 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5976 T75 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5977 T75 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 5977 T75 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 5990 T75 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 5995 T75 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 5996 T75 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 5999 T75 oass.SolrIndexSearcher.<init> Opening Searcher@66a22ed7 main
[junit4:junit4]   2> 5999 T75 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 6000 T75 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 6001 T75 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 6001 T75 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 6024 T75 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 6084 T92 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@66a22ed7 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 6091 T75 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6093 T75 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 6093 T75 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:7001/solr shardId:shard1
[junit4:junit4]   2> 6098 T75 oasc.ZkController.register We are http://127.0.0.1:7001/solr/collection1/ and leader is http://127.0.0.1:7000/solr/collection1/
[junit4:junit4]   2> 6098 T75 oasc.ZkController.checkRecovery WARNING Skipping recovery according to sys prop solrcloud.skip.autorecovery
[junit4:junit4]   2> 6099 T75 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6101 T75 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6104 T75 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 6104 T75 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 6105 T75 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 6105 T75 oasc.CoreContainer.<init> New CoreContainer 649127729
[junit4:junit4]   2> 6106 T75 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 6106 T75 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 6167 T75 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 6182 T75 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:57157/solr
[junit4:junit4]   2> 6183 T75 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57157/solr sessionTimeout=3000 watcher=org.apache.solr.common.cloud.ConnectionManager@467746a2
[junit4:junit4]   2> 6184 T93 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:57157
[junit4:junit4]   2> 6184 T75 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 6185 T93 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57157, initiating session
[junit4:junit4]   2> 6186 T77 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:21278
[junit4:junit4]   2> 6186 T77 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:21278
[junit4:junit4]   2> 6188 T79 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac3e1b9bc0004 with negotiated timeout 3000 for client /140.211.11.196:21278
[junit4:junit4]   2> 6188 T93 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:57157, sessionid = 0x13ac3e1b9bc0004, negotiated timeout = 3000
[junit4:junit4]   2> 6188 T94 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@467746a2 name:ZooKeeperConnection Watcher:127.0.0.1:57157/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 6188 T75 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 6189 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0004 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6191 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0004 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6200 T75 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:7002_solr
[junit4:junit4]   2> 6200 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0004 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:7002_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:7002_solr
[junit4:junit4]   2> 6201 T75 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:7002_solr
[junit4:junit4]   2> 6205 T86 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 6205 T91 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 6206 T75 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 6207 T75 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 6207 T75 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 6207 T75 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 6208 T75 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 6208 T75 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 6208 T75 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 6209 T75 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 6209 T75 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 6210 T75 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 6232 T75 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 6247 T75 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 6248 T75 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 6249 T75 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 6249 T75 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 6251 T75 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 6253 T75 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 6253 T75 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 6314 T75 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 6393 T75 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 6395 T75 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 6415 T75 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 6448 T87 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6449 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7001_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7001/solr"}
[junit4:junit4]   2> 6482 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7001_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7001/solr"}
[junit4:junit4]   2> 6494 T86 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 6494 T91 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 6494 T94 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 7037 T75 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 7065 T75 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 7070 T75 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 7102 T75 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 7108 T75 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 7114 T75 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 7116 T75 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 7118 T75 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7002/
[junit4:junit4]   2> 7118 T75 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@54db16d7
[junit4:junit4]   2> 7120 T75 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7002/index/
[junit4:junit4]   2> 7120 T75 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7002/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 7121 T75 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7002/index forceNew:false
[junit4:junit4]   2> 7123 T75 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@14dc1f90 lockFactory=org.apache.lucene.store.NativeFSLockFactory@677ec421),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 7124 T75 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 7124 T75 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 7128 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 7128 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 7129 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 7129 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 7130 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 7131 T75 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 7131 T75 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 7132 T75 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 7133 T75 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 7134 T75 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 7134 T75 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 7135 T75 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 7136 T75 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 7136 T75 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 7137 T75 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 7138 T75 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 7138 T75 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7139 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7140 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7140 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7141 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7142 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7143 T75 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7143 T75 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 7144 T75 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 7157 T75 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 7162 T75 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 7163 T75 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 7166 T75 oass.SolrIndexSearcher.<init> Opening Searcher@2cb056c8 main
[junit4:junit4]   2> 7166 T75 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 7167 T75 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 7167 T75 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 7168 T75 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 7202 T75 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 7262 T95 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2cb056c8 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 7269 T75 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 7272 T75 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 7272 T75 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:7002/solr shardId:shard1
[junit4:junit4]   2> 7276 T75 oasc.ZkController.register We are http://127.0.0.1:7002/solr/collection1/ and leader is http://127.0.0.1:7000/solr/collection1/
[junit4:junit4]   2> 7277 T75 oasc.ZkController.checkRecovery WARNING Skipping recovery according to sys prop solrcloud.skip.autorecovery
[junit4:junit4]   2> 7277 T75 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 7280 T75 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 7283 T75 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 7283 T75 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 7284 T75 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 7284 T75 oasc.CoreContainer.<init> New CoreContainer 1379550442
[junit4:junit4]   2> 7285 T75 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 7285 T75 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 7347 T75 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 7363 T75 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:57157/solr
[junit4:junit4]   2> 7363 T75 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57157/solr sessionTimeout=3000 watcher=org.apache.solr.common.cloud.ConnectionManager@6be7b7cd
[junit4:junit4]   2> 7364 T96 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:57157
[junit4:junit4]   2> 7364 T75 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7366 T96 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57157, initiating session
[junit4:junit4]   2> 7366 T77 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:35865
[junit4:junit4]   2> 7366 T77 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:35865
[junit4:junit4]   2> 7368 T79 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac3e1b9bc0005 with negotiated timeout 3000 for client /140.211.11.196:35865
[junit4:junit4]   2> 7368 T96 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:57157, sessionid = 0x13ac3e1b9bc0005, negotiated timeout = 3000
[junit4:junit4]   2> 7369 T97 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6be7b7cd name:ZooKeeperConnection Watcher:127.0.0.1:57157/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7369 T75 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7370 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0005 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7371 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0005 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7373 T75 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:7003_solr
[junit4:junit4]   2> 7373 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0005 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:7003_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:7003_solr
[junit4:junit4]   2> 7382 T75 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:7003_solr
[junit4:junit4]   2> 7385 T86 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 7385 T91 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 7385 T94 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 7386 T75 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 7386 T75 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 7387 T75 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 7387 T75 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 7387 T75 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 7388 T75 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 7388 T75 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 7388 T75 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 7389 T75 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 7389 T75 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 7411 T75 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 7418 T75 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 7418 T75 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 7419 T75 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 7420 T75 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 7421 T75 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 7423 T75 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 7424 T75 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 7480 T75 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 7513 T87 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 7514 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7002_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7002/solr"}
[junit4:junit4]   2> 7521 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7002_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7002/solr"}
[junit4:junit4]   2> 7534 T86 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 7534 T94 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 7534 T91 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 7534 T97 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 7552 T75 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 7554 T75 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 7570 T75 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 8182 T75 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 8197 T75 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 8201 T75 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 8226 T75 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8232 T75 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8237 T75 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8239 T75 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8241 T75 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7003/
[junit4:junit4]   2> 8241 T75 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@54db16d7
[junit4:junit4]   2> 8242 T75 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7003/index/
[junit4:junit4]   2> 8243 T75 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7003/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 8244 T75 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7003/index forceNew:false
[junit4:junit4]   2> 8246 T75 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@49fc1d7e lockFactory=org.apache.lucene.store.NativeFSLockFactory@5615a6e0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 8247 T75 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 8247 T75 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 8250 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 8251 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 8251 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 8251 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 8252 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 8253 T75 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 8253 T75 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 8254 T75 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 8255 T75 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 8255 T75 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 8256 T75 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 8257 T75 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 8257 T75 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 8258 T75 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 8258 T75 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 8259 T75 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 8259 T75 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8260 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8261 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8261 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8262 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8263 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8263 T75 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8264 T75 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 8265 T75 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 8277 T75 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 8281 T75 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 8283 T75 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 8286 T75 oass.SolrIndexSearcher.<init> Opening Searcher@68dd9142 main
[junit4:junit4]   2> 8286 T75 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 8287 T75 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 8287 T75 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 8288 T75 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 8337 T75 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 8492 T98 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@68dd9142 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 8498 T75 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8523 T75 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 8523 T75 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:7003/solr shardId:shard1
[junit4:junit4]   2> 8532 T75 oasc.ZkController.register We are http://127.0.0.1:7003/solr/collection1/ and leader is http://127.0.0.1:7000/solr/collection1/
[junit4:junit4]   2> 8532 T75 oasc.ZkController.checkRecovery WARNING Skipping recovery according to sys prop solrcloud.skip.autorecovery
[junit4:junit4]   2> 8533 T75 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8555 T75 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8557 T87 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8558 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7003_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7003/solr"}
[junit4:junit4]   2> 8559 T75 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 8559 T75 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 8560 T75 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 8560 T75 oasc.CoreContainer.<init> New CoreContainer 133269757
[junit4:junit4]   2> 8561 T75 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 8561 T75 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 8624 T75 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 8627 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7003_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7003/solr"}
[junit4:junit4]   2> 8640 T75 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:57157/solr
[junit4:junit4]   2> 8641 T75 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57157/solr sessionTimeout=3000 watcher=org.apache.solr.common.cloud.ConnectionManager@6ceb8bfa
[junit4:junit4]   2> 8642 T99 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:57157
[junit4:junit4]   2> 8642 T75 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 8643 T99 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57157, initiating session
[junit4:junit4]   2> 8643 T77 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:13862
[junit4:junit4]   2> 8644 T77 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:13862
[junit4:junit4]   2> 8664 T86 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 8664 T91 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 8664 T97 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 8664 T94 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 8665 T79 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac3e1b9bc0006 with negotiated timeout 3000 for client /140.211.11.196:13862
[junit4:junit4]   2> 8665 T99 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:57157, sessionid = 0x13ac3e1b9bc0006, negotiated timeout = 3000
[junit4:junit4]   2> 8665 T100 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6ceb8bfa name:ZooKeeperConnection Watcher:127.0.0.1:57157/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 8666 T75 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 8667 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0006 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8668 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0006 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8669 T75 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:7004_solr
[junit4:junit4]   2> 8670 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0006 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:7004_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:7004_solr
[junit4:junit4]   2> 8671 T75 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:7004_solr
[junit4:junit4]   2> 8674 T86 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 8674 T97 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 8674 T91 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 8674 T94 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 8675 T75 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 8676 T75 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 8676 T75 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 8676 T75 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 8677 T75 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 8677 T75 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 8677 T75 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 8678 T75 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 8678 T75 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 8678 T75 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 8699 T75 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 8706 T75 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 8706 T75 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 8707 T75 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 8708 T75 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 8709 T75 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 8711 T75 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 8711 T75 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 8764 T75 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 8840 T75 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 8842 T75 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 8856 T75 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 9409 T75 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 9422 T75 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 9425 T75 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 9445 T75 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9450 T75 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9454 T75 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9455 T75 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9457 T75 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7004/
[junit4:junit4]   2> 9457 T75 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@54db16d7
[junit4:junit4]   2> 9458 T75 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7004/index/
[junit4:junit4]   2> 9458 T75 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7004/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 9459 T75 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7004/index forceNew:false
[junit4:junit4]   2> 9461 T75 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@119cb48 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4fabbfd2),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 9461 T75 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 9461 T75 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 9463 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 9464 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 9464 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 9464 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 9465 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 9465 T75 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 9466 T75 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 9466 T75 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 9467 T75 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 9467 T75 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 9468 T75 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 9468 T75 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 9469 T75 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 9469 T75 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 9470 T75 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 9470 T75 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 9471 T75 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9471 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9472 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9472 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9473 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9473 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9474 T75 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9474 T75 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 9475 T75 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 9483 T75 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 9486 T75 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 9488 T75 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 9490 T75 oass.SolrIndexSearcher.<init> Opening Searcher@4242dfb main
[junit4:junit4]   2> 9490 T75 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 9490 T75 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 9491 T75 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 9491 T75 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 9523 T75 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 9576 T101 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4242dfb main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 9580 T75 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9582 T75 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 9582 T75 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:7004/solr shardId:shard1
[junit4:junit4]   2> 9586 T75 oasc.ZkController.register We are http://127.0.0.1:7004/solr/collection1/ and leader is http://127.0.0.1:7000/solr/collection1/
[junit4:junit4]   2> 9586 T75 oasc.ZkController.checkRecovery WARNING Skipping recovery according to sys prop solrcloud.skip.autorecovery
[junit4:junit4]   2> 9586 T75 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9593 T75 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9596 T75 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 9596 T75 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 9596 T75 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 9596 T75 oasc.CoreContainer.<init> New CoreContainer 108824254
[junit4:junit4]   2> 9597 T75 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 9597 T75 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 9636 T75 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 9646 T75 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:57157/solr
[junit4:junit4]   2> 9647 T75 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57157/solr sessionTimeout=3000 watcher=org.apache.solr.common.cloud.ConnectionManager@3111c576
[junit4:junit4]   2> 9648 T102 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:57157
[junit4:junit4]   2> 9648 T75 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 9649 T102 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57157, initiating session
[junit4:junit4]   2> 9649 T77 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:21843
[junit4:junit4]   2> 9649 T77 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:21843
[junit4:junit4]   2> 9651 T79 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac3e1b9bc0007 with negotiated timeout 3000 for client /140.211.11.196:21843
[junit4:junit4]   2> 9651 T102 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:57157, sessionid = 0x13ac3e1b9bc0007, negotiated timeout = 3000
[junit4:junit4]   2> 9652 T103 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3111c576 name:ZooKeeperConnection Watcher:127.0.0.1:57157/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 9652 T75 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 9653 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0007 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9654 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0007 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9655 T75 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:3333_solr
[junit4:junit4]   2> 9656 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0007 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:3333_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:3333_solr
[junit4:junit4]   2> 9665 T75 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:3333_solr
[junit4:junit4]   2> 9668 T86 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 9669 T91 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 9669 T97 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 9668 T94 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 9669 T100 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 9669 T75 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 9670 T75 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 9670 T75 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 9670 T75 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 9670 T75 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 9671 T75 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 9671 T75 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 9671 T75 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 9671 T75 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 9672 T75 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 9677 T87 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9677 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7004_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7004/solr"}
[junit4:junit4]   2> 9682 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7004_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7004/solr"}
[junit4:junit4]   2> 9692 T75 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 9693 T86 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 9693 T97 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 9693 T91 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 9693 T100 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 9693 T94 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 9693 T103 oaz.ClientCnxn$EventThread.processEvent SEVERE Error while calling watcher  java.lang.NullPointerException
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader$2.process(ZkStateReader.java:189)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:526)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
[junit4:junit4]   2> 
[junit4:junit4]   2> 9699 T75 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 9699 T75 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 9700 T75 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 9700 T75 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 9701 T75 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 9702 T75 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 9703 T75 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 9740 T75 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 9788 T75 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 9789 T75 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 9799 T75 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 10235 T75 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 10244 T75 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 10247 T75 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 10449 T75 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10454 T75 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10458 T75 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10459 T75 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10461 T75 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_3333/
[junit4:junit4]   2> 10461 T75 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@54db16d7
[junit4:junit4]   2> 10462 T75 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_3333/index/
[junit4:junit4]   2> 10462 T75 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_3333/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 10463 T75 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_3333/index forceNew:false
[junit4:junit4]   2> 10465 T75 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@19d0954e lockFactory=org.apache.lucene.store.NativeFSLockFactory@41c981b4),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 10465 T75 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 10466 T75 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 10468 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 10469 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 10469 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 10469 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 10470 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 10470 T75 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 10471 T75 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 10471 T75 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 10472 T75 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 10472 T75 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 10473 T75 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 10473 T75 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 10474 T75 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 10474 T75 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 10474 T75 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 10475 T75 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 10475 T75 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10476 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10476 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10477 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10477 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10478 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10478 T75 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10479 T75 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 10479 T75 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 10488 T75 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 10491 T75 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 10492 T75 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 10495 T75 oass.SolrIndexSearcher.<init> Opening Searcher@36800134 main
[junit4:junit4]   2> 10495 T75 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 10495 T75 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 10496 T75 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 10496 T75 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 10541 T75 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 10586 T104 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@36800134 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 10593 T75 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 10595 T75 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 10595 T75 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:3333/solr shardId:shard2
[junit4:junit4]   2> 10596 T75 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 10610 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0007 type:delete cxid:0x40 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 10611 T75 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 10612 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0007 type:create cxid:0x41 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10704 T87 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 10704 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:3333_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:3333/solr"}
[junit4:junit4]   2> 10722 T91 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 10722 T86 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 10723 T94 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 10723 T100 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 107

[...truncated too long message...]

ve_errors=0}
[junit4:junit4]   2> 7222870 T75 oasc.SolrCore.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 7222870 T75 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 7222871 T75 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 7222871 T75 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 7223297 T102 oaz.ClientCnxn$SendThread.run Client session timed out, have not heard from server in 3000ms for sessionid 0x13ac3e1b9bc0007, closing socket connection and attempting reconnect
[junit4:junit4]   2> 7223342 T75 oaz.ZooKeeper.close Session: 0x13ac3e1b9bc0006 closed
[junit4:junit4]   2> 7223342 T75 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=108824254
[junit4:junit4]   2> 7223343 T75 oasc.SolrCore.close SEVERE Too many close [count:-1] on org.apache.solr.core.SolrCore@5bb8ad39. Please report this exception to solr-user@lucene.apache.org
[junit4:junit4]   2> 7223371 T74 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
[junit4:junit4]   2> 	   1) Thread[id=102, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[74596681D419C368]-SendThread(localhost:57157), state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	        at java.lang.Thread.sleep(Native Method)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1038)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1101)
[junit4:junit4]   2> 	   2) Thread[id=75, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[74596681D419C368], state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	        at java.lang.Object.wait(Native Method)
[junit4:junit4]   2> 	        at java.lang.Object.wait(Object.java:503)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1345)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn.close(ClientCnxn.java:1323)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ZooKeeper.close(ZooKeeper.java:537)
[junit4:junit4]   2> 	        at org.apache.solr.common.cloud.SolrZooKeeper.close(SolrZooKeeper.java:91)
[junit4:junit4]   2> 	        at org.apache.solr.common.cloud.SolrZkClient.closeKeeper(SolrZkClient.java:573)
[junit4:junit4]   2> 	        at org.apache.solr.common.cloud.SolrZkClient.close(SolrZkClient.java:542)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.ZkController.close(ZkController.java:319)
[junit4:junit4]   2> 	        at org.apache.solr.core.CoreContainer.shutdown(CoreContainer.java:603)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.LeaderElectionIntegrationTest.tearDown(LeaderElectionIntegrationTest.java:289)
[junit4:junit4]   2> 	        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4]   2> 	        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4:junit4]   2> 	        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4:junit4]   2> 	        at java.lang.reflect.Method.invoke(Method.java:601)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:795)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene40, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=el_GR, timezone=America/Cancun
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=6,free=90418984,total=145752064
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestFaceting, TestGroupingSearch, DistanceFunctionTest, LeaderElectionIntegrationTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderElectionIntegrationTest -Dtests.seed=74596681D419C368 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=el_GR -Dtests.timezone=America/Cancun -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] ERROR   0.00s J0 | LeaderElectionIntegrationTest (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([74596681D419C368]:0)
[junit4:junit4] Completed on J0 in 7223.42s, 1 test, 2 errors <<< FAILURES!

[...truncated 174 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:335: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/module-build.xml:61: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:1163: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:827: There were test failures: 233 suites, 964 tests, 3 suite-level errors, 1 error, 850 ignored

Total time: 137 minutes 58 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure

[JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 608 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.

Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/608/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.request.TestRemoteStreaming

Error Message:
4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming:     1) Thread[id=297, name=qtp1321261771-297 Acceptor2 SelectChannelConnector@0.0.0.0:58147, state=RUNNABLE, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=298, name=qtp1321261771-298 Acceptor3 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=296, name=qtp1321261771-296 Acceptor1 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=295, name=qtp1321261771-295 Acceptor0 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming: 
   1) Thread[id=297, name=qtp1321261771-297 Acceptor2 SelectChannelConnector@0.0.0.0:58147, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=298, name=qtp1321261771-298 Acceptor3 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=296, name=qtp1321261771-296 Acceptor1 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=295, name=qtp1321261771-295 Acceptor0 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([8E53F6B1A0D0F2CF]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.request.TestRemoteStreaming

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=297, name=qtp1321261771-297 Acceptor2 SelectChannelConnector@0.0.0.0:58147, state=RUNNABLE, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=298, name=qtp1321261771-298 Acceptor3 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=296, name=qtp1321261771-296 Acceptor1 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=295, name=qtp1321261771-295 Acceptor0 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=297, name=qtp1321261771-297 Acceptor2 SelectChannelConnector@0.0.0.0:58147, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=298, name=qtp1321261771-298 Acceptor3 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=296, name=qtp1321261771-296 Acceptor1 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=295, name=qtp1321261771-295 Acceptor0 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([8E53F6B1A0D0F2CF]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.update.SolrCmdDistributorTest

Error Message:
20 threads leaked from SUITE scope at org.apache.solr.update.SolrCmdDistributorTest:     1) Thread[id=32, name=qtp1160133474-32 Acceptor0 SelectChannelConnector@0.0.0.0:11368, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=62, name=qtp888776338-62 Acceptor2 SelectChannelConnector@0.0.0.0:13188, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=19, name=qtp2137918572-19 Acceptor1 SelectChannelConnector@0.0.0.0:61902, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=74, name=qtp1946214468-74 Acceptor0 SelectChannelConnector@0.0.0.0:65108, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=60, name=qtp888776338-60 Acceptor0 SelectChannelConnector@0.0.0.0:13188, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=48, name=qtp544144105-48 Acceptor2 SelectChannelConnector@0.0.0.0:53691, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=76, name=qtp1946214468-76 Acceptor2 SelectChannelConnector@0.0.0.0:65108, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=61, name=qtp888776338-61 Acceptor1 SelectChannelConnector@0.0.0.0:13188, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=77, name=qtp1946214468-77 Acceptor3 SelectChannelConnector@0.0.0.0:65108, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=63, name=qtp888776338-63 Acceptor3 SelectChannelConnector@0.0.0.0:13188, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=20, name=qtp2137918572-20 Acceptor2 SelectChannelConnector@0.0.0.0:61902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=46, name=qtp544144105-46 Acceptor0 SelectChannelConnector@0.0.0.0:53691, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=49, name=qtp544144105-49 Acceptor3 SelectChannelConnector@0.0.0.0:53691, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=33, name=qtp1160133474-33 Acceptor1 SelectChannelConnector@0.0.0.0:11368, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=47, name=qtp544144105-47 Acceptor1 SelectChannelConnector@0.0.0.0:53691, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=35, name=qtp1160133474-35 Acceptor3 SelectChannelConnector@0.0.0.0:11368, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=75, name=qtp1946214468-75 Acceptor1 SelectChannelConnector@0.0.0.0:65108, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=21, name=qtp2137918572-21 Acceptor3 SelectChannelConnector@0.0.0.0:61902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=18, name=qtp2137918572-18 Acceptor0 SelectChannelConnector@0.0.0.0:61902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=34, name=qtp1160133474-34 Acceptor2 SelectChannelConnector@0.0.0.0:11368, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 20 threads leaked from SUITE scope at org.apache.solr.update.SolrCmdDistributorTest: 
   1) Thread[id=32, name=qtp1160133474-32 Acceptor0 SelectChannelConnector@0.0.0.0:11368, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=62, name=qtp888776338-62 Acceptor2 SelectChannelConnector@0.0.0.0:13188, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=19, name=qtp2137918572-19 Acceptor1 SelectChannelConnector@0.0.0.0:61902, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=74, name=qtp1946214468-74 Acceptor0 SelectChannelConnector@0.0.0.0:65108, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=60, name=qtp888776338-60 Acceptor0 SelectChannelConnector@0.0.0.0:13188, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=48, name=qtp544144105-48 Acceptor2 SelectChannelConnector@0.0.0.0:53691, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=76, name=qtp1946214468-76 Acceptor2 SelectChannelConnector@0.0.0.0:65108, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=61, name=qtp888776338-61 Acceptor1 SelectChannelConnector@0.0.0.0:13188, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=77, name=qtp1946214468-77 Acceptor3 SelectChannelConnector@0.0.0.0:65108, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=63, name=qtp888776338-63 Acceptor3 SelectChannelConnector@0.0.0.0:13188, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=20, name=qtp2137918572-20 Acceptor2 SelectChannelConnector@0.0.0.0:61902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=46, name=qtp544144105-46 Acceptor0 SelectChannelConnector@0.0.0.0:53691, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=49, name=qtp544144105-49 Acceptor3 SelectChannelConnector@0.0.0.0:53691, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=33, name=qtp1160133474-33 Acceptor1 SelectChannelConnector@0.0.0.0:11368, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=47, name=qtp544144105-47 Acceptor1 SelectChannelConnector@0.0.0.0:53691, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=35, name=qtp1160133474-35 Acceptor3 SelectChannelConnector@0.0.0.0:11368, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=75, name=qtp1946214468-75 Acceptor1 SelectChannelConnector@0.0.0.0:65108, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=21, name=qtp2137918572-21 Acceptor3 SelectChannelConnector@0.0.0.0:61902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=18, name=qtp2137918572-18 Acceptor0 SelectChannelConnector@0.0.0.0:61902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=34, name=qtp1160133474-34 Acceptor2 SelectChannelConnector@0.0.0.0:11368, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([8E53F6B1A0D0F2CF]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.update.SolrCmdDistributorTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=32, name=qtp1160133474-32 Acceptor0 SelectChannelConnector@0.0.0.0:11368, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=19, name=qtp2137918572-19 Acceptor1 SelectChannelConnector@0.0.0.0:61902, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=62, name=qtp888776338-62 Acceptor2 SelectChannelConnector@0.0.0.0:13188, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=60, name=qtp888776338-60 Acceptor0 SelectChannelConnector@0.0.0.0:13188, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=74, name=qtp1946214468-74 Acceptor0 SelectChannelConnector@0.0.0.0:65108, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=48, name=qtp544144105-48 Acceptor2 SelectChannelConnector@0.0.0.0:53691, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=76, name=qtp1946214468-76 Acceptor2 SelectChannelConnector@0.0.0.0:65108, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=61, name=qtp888776338-61 Acceptor1 SelectChannelConnector@0.0.0.0:13188, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=20, name=qtp2137918572-20 Acceptor2 SelectChannelConnector@0.0.0.0:61902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=63, name=qtp888776338-63 Acceptor3 SelectChannelConnector@0.0.0.0:13188, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=77, name=qtp1946214468-77 Acceptor3 SelectChannelConnector@0.0.0.0:65108, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=46, name=qtp544144105-46 Acceptor0 SelectChannelConnector@0.0.0.0:53691, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=49, name=qtp544144105-49 Acceptor3 SelectChannelConnector@0.0.0.0:53691, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=33, name=qtp1160133474-33 Acceptor1 SelectChannelConnector@0.0.0.0:11368, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=47, name=qtp544144105-47 Acceptor1 SelectChannelConnector@0.0.0.0:53691, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=35, name=qtp1160133474-35 Acceptor3 SelectChannelConnector@0.0.0.0:11368, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=75, name=qtp1946214468-75 Acceptor1 SelectChannelConnector@0.0.0.0:65108, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=21, name=qtp2137918572-21 Acceptor3 SelectChannelConnector@0.0.0.0:61902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=18, name=qtp2137918572-18 Acceptor0 SelectChannelConnector@0.0.0.0:61902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=34, name=qtp1160133474-34 Acceptor2 SelectChannelConnector@0.0.0.0:11368, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=32, name=qtp1160133474-32 Acceptor0 SelectChannelConnector@0.0.0.0:11368, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=19, name=qtp2137918572-19 Acceptor1 SelectChannelConnector@0.0.0.0:61902, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=62, name=qtp888776338-62 Acceptor2 SelectChannelConnector@0.0.0.0:13188, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=60, name=qtp888776338-60 Acceptor0 SelectChannelConnector@0.0.0.0:13188, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=74, name=qtp1946214468-74 Acceptor0 SelectChannelConnector@0.0.0.0:65108, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=48, name=qtp544144105-48 Acceptor2 SelectChannelConnector@0.0.0.0:53691, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=76, name=qtp1946214468-76 Acceptor2 SelectChannelConnector@0.0.0.0:65108, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=61, name=qtp888776338-61 Acceptor1 SelectChannelConnector@0.0.0.0:13188, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=20, name=qtp2137918572-20 Acceptor2 SelectChannelConnector@0.0.0.0:61902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=63, name=qtp888776338-63 Acceptor3 SelectChannelConnector@0.0.0.0:13188, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=77, name=qtp1946214468-77 Acceptor3 SelectChannelConnector@0.0.0.0:65108, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=46, name=qtp544144105-46 Acceptor0 SelectChannelConnector@0.0.0.0:53691, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=49, name=qtp544144105-49 Acceptor3 SelectChannelConnector@0.0.0.0:53691, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=33, name=qtp1160133474-33 Acceptor1 SelectChannelConnector@0.0.0.0:11368, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=47, name=qtp544144105-47 Acceptor1 SelectChannelConnector@0.0.0.0:53691, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=35, name=qtp1160133474-35 Acceptor3 SelectChannelConnector@0.0.0.0:11368, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=75, name=qtp1946214468-75 Acceptor1 SelectChannelConnector@0.0.0.0:65108, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=21, name=qtp2137918572-21 Acceptor3 SelectChannelConnector@0.0.0.0:61902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=18, name=qtp2137918572-18 Acceptor0 SelectChannelConnector@0.0.0.0:61902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=34, name=qtp1160133474-34 Acceptor2 SelectChannelConnector@0.0.0.0:11368, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([8E53F6B1A0D0F2CF]:0)




Build Log:
[...truncated 8368 lines...]
[junit4:junit4] Suite: org.apache.solr.update.SolrCmdDistributorTest
[junit4:junit4]   2> 213 T11 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> 674 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 982 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:61902
[junit4:junit4]   2> 1127 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1128 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 1129 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 1766 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1770 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1771 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 1803 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 1808 T11 oasc.CoreContainer.<init> New CoreContainer 665935931
[junit4:junit4]   2> 1809 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 1810 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 1897 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 1980 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 1981 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 1983 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 1984 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 2067 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 2234 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 2264 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 2338 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 4426 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 4476 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 4482 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 4920 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4942 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4948 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4970 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5078 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/control/data/
[junit4:junit4]   2> 5078 T11 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 5142 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/control/data/index/
[junit4:junit4]   2> 5143 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1351879716743/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5232 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/control/data/index forceNew:false
[junit4:junit4]   2> 5655 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@555a7085 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5dcd1c83; maxCacheMB=0.2607421875 maxMergeSizeMB=0.7861328125)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5656 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 6654 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 6678 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 6715 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 6721 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 6897 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 6997 T11 oass.SolrIndexSearcher.<init> Opening Searcher@53a46b20 main
[junit4:junit4]   2> 6998 T11 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 7117 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 7117 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 7215 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 7216 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 7217 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 7217 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 7218 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 7219 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 7219 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 7220 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 7220 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 7270 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 7791 T26 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@53a46b20 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 7793 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 7794 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 7794 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 7845 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 7869 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 7872 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:11368
[junit4:junit4]   2> 7873 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 7874 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 7875 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 7908 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 7909 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 7910 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 7910 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 7911 T11 oasc.CoreContainer.<init> New CoreContainer 97269771
[junit4:junit4]   2> 7912 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 7913 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 7958 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 7991 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 7993 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 7995 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 7996 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 8043 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 8114 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 8115 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 8169 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 8801 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 8818 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 8824 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 8848 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8854 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8859 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8864 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8868 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard0/data/
[junit4:junit4]   2> 8869 T11 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 8871 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard0/data/index/
[junit4:junit4]   2> 8873 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard0/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 8897 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard0/data/index forceNew:false
[junit4:junit4]   2> 8906 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@614c0482 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c7e3f6b),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 8906 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 8908 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 8909 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 8909 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 8910 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 8919 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 8924 T11 oass.SolrIndexSearcher.<init> Opening Searcher@41e9d96a main
[junit4:junit4]   2> 8925 T11 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 8927 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 8927 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 8928 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 8929 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 8929 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 8930 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 8930 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 8931 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 8931 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 8932 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 8932 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 8933 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 8946 T40 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@41e9d96a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 8946 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 8947 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 8948 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 8949 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 8960 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 8963 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:53691
[junit4:junit4]   2> 8964 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 8965 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 8965 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 8999 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 9000 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 9000 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 9001 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 9002 T11 oasc.CoreContainer.<init> New CoreContainer 1533846421
[junit4:junit4]   2> 9003 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 9003 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 9039 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 9059 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 9060 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 9062 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 9063 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 9110 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 9170 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 9171 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 9225 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 9831 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 9845 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 9848 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 9871 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9876 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9881 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9884 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9886 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard1/data/
[junit4:junit4]   2> 9887 T11 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 9888 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard1/data/index/
[junit4:junit4]   2> 9889 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard1/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 10008 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard1/data/index forceNew:false
[junit4:junit4]   2> 10021 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.MMapDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/index4945549455tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@3c417f94),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 10022 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 10023 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 10024 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 10025 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 10025 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 10036 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 10056 T11 oass.SolrIndexSearcher.<init> Opening Searcher@31eb6efa main
[junit4:junit4]   2> 10056 T11 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 10057 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 10058 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 10059 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 10059 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 10060 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 10060 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 10061 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 10061 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 10062 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 10062 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 10063 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 10063 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 10080 T54 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@31eb6efa main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 10079 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 10080 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 10081 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 10082 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 10094 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 10096 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:13188
[junit4:junit4]   2> 10097 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 10098 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 10098 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 10136 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 10137 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 10137 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 10138 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 10138 T11 oasc.CoreContainer.<init> New CoreContainer 274270528
[junit4:junit4]   2> 10139 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 10140 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 10172 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 10191 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 10192 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 10193 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 10194 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 10241 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 10291 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 10293 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 10332 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 10926 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 10941 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 10945 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 10965 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10971 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10976 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10978 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10981 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard2/data/
[junit4:junit4]   2> 10982 T11 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 10983 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard2/data/index/
[junit4:junit4]   2> 10984 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard2/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 10985 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard2/data/index forceNew:false
[junit4:junit4]   2> 10989 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5b895852 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ae51802),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 10989 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 10991 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 10991 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 10992 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 10993 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 11006 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 11013 T11 oass.SolrIndexSearcher.<init> Opening Searcher@64f8145f main
[junit4:junit4]   2> 11014 T11 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 11015 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 11016 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 11016 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 11017 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 11017 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 11018 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 11018 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 11019 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 11019 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 11020 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 11020 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 11021 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 11034 T68 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@64f8145f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 11034 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 11035 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 11035 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 11036 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 11047 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 11050 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:65108
[junit4:junit4]   2> 11050 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 11051 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 11052 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 11093 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 11094 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 11094 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 11095 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 11095 T11 oasc.CoreContainer.<init> New CoreContainer 2108989886
[junit4:junit4]   2> 11096 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 11097 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 11129 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 11149 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 11150 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 11152 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 11153 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 11201 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 11258 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 11259 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 11293 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 11849 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 11865 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 11869 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 11889 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 11894 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 11900 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 11902 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 11904 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard3/data/
[junit4:junit4]   2> 11905 T11 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 11906 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard3/data/index/
[junit4:junit4]   2> 11907 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard3/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 11908 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard3/data/index forceNew:false
[junit4:junit4]   2> 11911 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@26fb5989 lockFactory=org.apache.lucene.store.NativeFSLockFactory@128efd57),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 11912 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 11913 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 11914 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 11914 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 11915 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 11925 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 11931 T11 oass.SolrIndexSearcher.<init> Opening Searcher@5aa23fc8 main
[junit4:junit4]   2> 11931 T11 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 11932 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 11933 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 11934 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 11934 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 11935 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 11935 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 11936 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 11936 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 11937 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 11937 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 11937 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 11938 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 11951 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 11951 T82 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5aa23fc8 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 11952 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 11953 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 11954 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C0 name=collection1 org.apache.solr.core.SolrCore@281c7736
[junit4:junit4]   2> 13515 T22 C0 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@555a7085 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5dcd1c83; maxCacheMB=0.2607421875 maxMergeSizeMB=0.7861328125)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 13520 T22 C0 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 13579 T22 C0 oass.SolrIndexSearcher.<init> Opening Searcher@6201a5c realtime
[junit4:junit4]   2> 13580 T22 C0 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 13588 T22 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1417548639409012736)} 0 745
[junit4:junit4]   2> ASYNC  NEW_CORE C1 name=collection1 org.apache.solr.core.SolrCore@463fead8
[junit4:junit4]   2> 13792 T38 C1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@614c0482 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c7e3f6b),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 13793 T38 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 13795 T38 C1 oass.SolrIndexSearcher.<init> Opening Searcher@3316d58c realtime
[junit4:junit4]   2> 13796 T38 C1 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 13797 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1417548640092684288)} 0 9
[junit4:junit4]   2> ASYNC  NEW_CORE C2 name=collection1 org.apache.solr.core.SolrCore@2e5d21b2
[junit4:junit4]   2> 13808 T53 C2 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.MMapDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/index4945549455tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@3c417f94),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 13808 T53 C2 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 13810 T53 C2 oass.SolrIndexSearcher.<init> Opening Searcher@51c3358f realtime
[junit4:junit4]   2> 13810 T53 C2 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 13811 T53 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1417548640109461504)} 0 8
[junit4:junit4]   2> ASYNC  NEW_CORE C3 name=collection1 org.apache.solr.core.SolrCore@532ba0c2
[junit4:junit4]   2> 13820 T66 C3 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5b895852 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ae51802),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 13821 T66 C3 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 13822 T66 C3 oass.SolrIndexSearcher.<init> Opening Searcher@37f4ff60 realtime
[junit4:junit4]   2> 13823 T66 C3 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 13823 T66 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1417548640124141568)} 0 6
[junit4:junit4]   2> ASYNC  NEW_CORE C4 name=collection1 org.apache.solr.core.SolrCore@74a34961
[junit4:junit4]   2> 13835 T80 C4 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@26fb5989 lockFactory=org.apache.lucene.store.NativeFSLockFactory@128efd57),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 13836 T80 C4 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 13838 T80 C4 oass.SolrIndexSearcher.<init> Opening Searcher@84ffc19 realtime
[junit4:junit4]   2> 13839 T80 C4 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 13840 T80 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1417548640138821632)} 0 9
[junit4:junit4]   2> 13859 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16
[junit4:junit4]   2> 14406 T24 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1 (1417548640271990784)]} 0 461
[junit4:junit4]   2> 14414 T11 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:61902/solr/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4]   2> 14437 T25 C0 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 14555 T25 C0 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@555a7085 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5dcd1c83; maxCacheMB=0.2607421875 maxMergeSizeMB=0.7861328125)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@555a7085 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5dcd1c83; maxCacheMB=0.2607421875 maxMergeSizeMB=0.7861328125)),segFN=segments_2,generation=2,filenames=[_0_Lucene40_0.tim, _0.fnm, _0_Lucene40_0.frq, _0_nrm.cfe, segments_2, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene40_0.prx, _0.fdt, _0_Lucene40_0.tip]
[junit4:junit4]   2> 14555 T25 C0 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 14682 T25 C0 oass.SolrIndexSearcher.<init> Opening Searcher@2554373f main
[junit4:junit4]   2> 14683 T25 C0 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 14687 T25 C0 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 14694 T26 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2554373f main{StandardDirectoryReader(segments_2:3:nrt _0(4.1):C1)}
[junit4:junit4]   2> 14695 T25 C0 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&commit_end_point=true&expungeDeletes=false&wt=javabin&softCommit=false&version=2} {commit=} 0 258
[junit4:junit4]   2> 14840 T24 C0 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=1 status=0 QTime=118 
[junit4:junit4]   2> 14879 T22 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&commit_end_point=true&version=2} {add=[2 (1417548641231437824), 3 (1417548641234583552)]} 0 5
[junit4:junit4]   2> 14881 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&commit_end_point=true&version=2} {add=[2 (1417548641230389248), 3 (1417548641233534976), 4 (1417548641236680704)]} 0 9
[junit4:junit4]   2> 14883 T11 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:61902/solr/, StdNode: http://127.0.0.1:11368/solr/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4]   2> 14887 T24 C0 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 14887 T37 C1 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 14893 T37 C1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@614c0482 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c7e3f6b),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@614c0482 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c7e3f6b),segFN=segments_2,generation=2,filenames=[_0_Lucene40_0.tim, _0.fnm, _0_Lucene40_0.frq, _0_nrm.cfe, segments_2, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene40_0.prx, _0.fdt, _0_Lucene40_0.tip]
[junit4:junit4]   2> 14893 T37 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 14896 T37 C1 oass.SolrIndexSearcher.<init> Opening Searcher@d8009ee main
[junit4:junit4]   2> 14896 T37 C1 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 14897 T37 C1 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 14897 T40 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@d8009ee main{StandardDirectoryReader(segments_2:3:nrt _0(4.1):C3)}
[junit4:junit4]   2> 14899 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&commit_end_point=true&expungeDeletes=false&wt=javabin&softCommit=false&version=2} {commit=} 0 12
[junit4:junit4]   2> 14937 T24 C0 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@555a7085 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5dcd1c83; maxCacheMB=0.2607421875 maxMergeSizeMB=0.7861328125)),segFN=segments_2,generation=2,filenames=[_0_Lucene40_0.tim, _0.fnm, _0_Lucene40_0.frq, _0_nrm.cfe, segments_2, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene40_0.prx, _0.fdt, _0_Lucene40_0.tip]
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@555a7085 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5dcd1c83; maxCacheMB=0.2607421875 maxMergeSizeMB=0.7861328125)),segFN=segments_3,generation=3,filenames=[_1_Lucene40_0.tim, _1_nrm.cfs, _0_Lucene40_0.tim, _0.fnm, _1_Lucene40_0.frq, _0_nrm.cfs, _0_Lucene40_0.prx, _1.fnm, _1_Lucene40_0.prx, _1_Lucene40_0.tip, _1_nrm.cfe, _0_Lucene40_0.tip, _1.fdx, _0_Lucene40_0.frq, _1.fdt, _1.si, _0_nrm.cfe, _0.fdx, _0.si, segments_3, _0.fdt]
[junit4:junit4]   2> 14937 T24 C0 oasc.SolrDeletionPolicy.updateCommits newest commit = 3
[junit4:junit4]   2> 14943 T24 C0 oass.SolrIndexSearcher.<init> Opening Searcher@50df92ed main
[junit4:junit4]   2> 14944 T24 C0 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 14945 T26 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@50df92ed main{StandardDirectoryReader(segments_3:5:nrt _0(4.1):C1 _1(4.1):C2)}
[junit4:junit4]   2> 14945 T24 C0 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 14946 T24 C0 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&commit_end_point=true&expungeDeletes=false&wt=javabin&softCommit=false&version=2} {commit=} 0 59
[junit4:junit4]   2> 14950 T25 C0 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=3 status=0 QTime=1 
[junit4:junit4]   2> 14961 T38 C1 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=3 status=0 QTime=1 
[junit4:junit4]   2> 14970 T24 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&commit_end_point=true&version=2} {delete=[2 (-1417548641331052544)]} 0 1
[junit4:junit4]   2> 14971 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&commit_end_point=true&version=2} {delete=[2 (-1417548641332101120)]} 0 2
[junit4:junit4]   2> 14972 T11 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:61902/solr/, StdNode: http://127.0.0.1:11368/solr/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4]   2> 14976 T22 C0 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 14976 T37 C1 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 14991 T37 C1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@614c0482 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c7e3f6b),segFN=segments_2,generation=2,filenames=[_0_Lucene40_0.tim, _0.fnm, _0_Lucene40_0.frq, _0_nrm.cfe, segments_2, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene40_0.prx, _0.fdt, _0_Lucene40_0.tip]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@614c0482 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c7e3f6b),segFN=segments_3,generation=3,filenames=[_0_Lucene40_0.tim, _0.fnm, _0_Lucene40_0.frq, _0_nrm.cfe, _0_1.del, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene40_0.prx, _0.fdt, segments_3, _0_Lucene40_0.tip]
[junit4:junit4]   2> 14991 T22 C0 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@555a7085 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5dcd1c83; maxCacheMB=0.2607421875 maxMergeSizeMB=0.7861328125)),segFN=segments_3,generation=3,filenames=[_1_Lucene40_0.tim, _1_nrm.cfs, _0_Lucene40_0.tim, _0.fnm, _1_Lucene40_0.frq, _0_nrm.cfs, _0_Lucene40_0.prx, _1.fnm, _1_Lucene40_0.prx, _1_Lucene40_0.tip, _1_nrm.cfe, _0_Lucene40_0.tip, _1.fdx, _0_Lucene40_0.frq, _1.fdt, _1.si, _0_nrm.cfe, _0.fdx, _0.si, segments_3, _0.fdt]
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@555a7085 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5dcd1c83; maxCacheMB=0.2607421875 maxMergeSizeMB=0.7861328125)),segFN=segments_4,generation=4,filenames=[_1_Lucene40_0.tim, _1_nrm.cfs, _0_Lucene40_0.tim, _0.fnm, _1_Lucene40_0.frq, _0_nrm.cfs, _1_1.del, _0_Lucene40_0.prx, _1.fnm, _1_Lucene40_0.prx, _1_Lucene40_0.tip, _1_nrm.cfe, _0_Lucene40_0.tip, _1.fdx, _0_Lucene40_0.frq, _1.fdt, _1.si, _0_nrm.cfe, _0.fdx, _0.si, _0.fdt, segments_4]
[junit4:junit4]   2> 14991 T37 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 3
[junit4:junit4]   2> 14992 T22 C0 oasc.SolrDeletionPolicy.updateCommits newest commit = 4
[junit4:junit4]   2> 14993 T37 C1 oass.SolrIndexSearcher.<init> Opening Searcher@4dc679c8 main
[junit4:junit4]   2> 14993 T37 C1 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 14994 T40 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4dc679c8 main{StandardDirectoryReader(segments_3:4:nrt _0(4.1):C3/1)}
[junit4:junit4]   2> 14994 T37 C1 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 14995 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&commit_end_point=true&expungeDeletes=false&wt=javabin&softCommit=false&version=2} {commit=} 0 19
[junit4:junit4]   2> 14997 T22 C0 oass.SolrIndexSearcher.<init> Opening Searcher@2ee5d440 main
[junit4:junit4]   2> 14997 T22 C0 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 14998 T26 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2ee5d440 main{StandardDirectoryReader(segments_4:6:nrt _0(4.1):C1 _1(4.1):C2/1)}
[junit4:junit4]   2> 14998 T22 C0 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 14999 T22 C0 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&commit_end_point=true&expungeDeletes=false&wt=javabin&softCommit=false&version=2} {commit=} 0 23
[junit4:junit4]   2> 15003 T25 C0 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=2 status=0 QTime=1 
[junit4:junit4]   2> 15008 T38 C1 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=2 status=0 QTime=1 
[junit4:junit4]   2> 15012 T37 C1 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15399 T37 C1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@614c0482 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c7e3f6b),segFN=segments_3,generation=3,filenames=[_0_Lucene40_0.tim, _0.fnm, _0_Lucene40_0.frq, _0_nrm.cfe, _0_1.del, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene40_0.prx, _0.fdt, segments_3, _0_Lucene40_0.tip]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@614c0482 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c7e3f6b),segFN=segments_4,generation=4,filenames=[_1_Lucene40_0.tim, _1.fdx, _1_nrm.cfs, _1.fdt, _1.si, _1_Lucene40_0.frq, _1.fnm, _1_Lucene40_0.prx, _1_Lucene40_0.tip, _1_nrm.cfe, segments_4]
[junit4:junit4]   2> 15400 T37 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 4
[junit4:junit4]   2> 15405 T37 C1 oass.SolrIndexSearcher.<init> Opening Searcher@19b161e6 main
[junit4:junit4]   2> 15405 T37 C1 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 15406 T37 C1 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15406 T40 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@19b161e6 main{StandardDirectoryReader(segments_4:6:nrt _1(4.1):C2)}
[junit4:junit4]   2> 15408 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={optimize=true&maxSegments=1&waitSearcher=true&wt=javabin&version=2} {optimize=} 0 396
[junit4:junit4]   2> 15412 T51 C2 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15414 T51 C2 oass.SolrIndexSearcher.<init> Opening Searcher@393ddf54 main
[junit4:junit4]   2> 15414 T51 C2 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 15415 T54 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@393ddf54 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4]   2> 15415 T51 C2 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15416 T51 C2 UPDATE [collection1] webapp=/solr path=/update params={optimize=true&maxSegments=1&waitSearcher=true&wt=javabin&version=2} {optimize=} 0 4
[junit4:junit4]   2> 15421 T67 C3 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15422 T67 C3 oass.SolrIndexSearcher.<init> Opening Searcher@59d3b486 main
[junit4:junit4]   2> 15423 T67 C3 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 15423 T68 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@59d3b486 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4]   2> 15424 T67 C3 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15424 T67 C3 UPDATE [collection1] webapp=/solr path=/update params={optimize=true&maxSegments=1&waitSearcher=true&wt=javabin&version=2} {optimize=} 0 3
[junit4:junit4]   2> 15428 T80 C4 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15430 T80 C4 oass.SolrIndexSearcher.<init> Opening Searcher@1d2ab101 main
[junit4:junit4]   2> 15430 T80 C4 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 15431 T82 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1d2ab101 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4]   2> 15431 T80 C4 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15432 T80 C4 UPDATE [collection1] webapp=/solr path=/update params={optimize=true&maxSegments=1&waitSearcher=true&wt=javabin&version=2} {optimize=} 0 4
[junit4:junit4]   2> 15703 T78 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[6 (1417548641867923456), 8 (1417548641876312064), 10 (1417548641881554944), 14 (1417548642058764288), 15 (1417548642061910016), 17 (1417548642067152896), 19 (1417548642072395776), 21 (1417548642083930112), 22 (1417548642087075840), 24 (1417548642090221568)]} 0 221
[junit4:junit4]   2> 16053 T65 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[7 (1417548641870020608), 8 (1417548641879457792), 11 (1417548642067152896), 12 (1417548642072395777), 14 (1417548642078687233), 17 (1417548642091270144), 19 (1417548642096513024), 22 (1417548642433105920), 23 (1417548642455126017), 25 (1417548642463514624)]} 0 571
[junit4:junit4]   2> 16069 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[33 (1417548641876312065), 35 (1417548641904623616), 37 (1417548641907769344), 38 (1417548642061910016), 40 (1417548642069250048), 41 (1417548642071347201), 43 (1417548642075541504), 49 (1417548642078687232), 52 (1417548642089172993), 53 (1417548642101755904)]} 0 581
[junit4:junit4]   2> 16070 T98 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[66 (1417548641879457792), 68 (1417548642071347200), 69 (1417548642077638656), 70 (1417548642087075840), 71 (1417548642090221568), 72 (1417548642092318720), 73 (1417548642101755904), 75 (1417548642434154497), 76 (1417548642444640256), 78 (1417548642446737408)]} 0 578
[junit4:junit4]   2> 16073 T105 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[83 (1417548642068201472), 85 (1417548642075541505), 87 (1417548642082881536), 89 (1417548642086027264), 90 (1417548642089172993), 91 (1417548642093367296), 92 (1417548642103853056), 94 (1417548642434154496), 95 (1417548642437300224), 97 (1417548642443591680)]} 0 405
[junit4:junit4]   2> 16070 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[6 (1417548641871069184), 8 (1417548641876312064), 9 (1417548641879457792), 15 (1417548641881554944), 19 (1417548642070298624), 23 (1417548642076590081), 24 (1417548642087075840), 25 (1417548642091270144), 29 (1417548642093367296), 30 (1417548642433105920)]} 0 586
[junit4:junit4]   2> 16073 T52 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[5 (1417548641868972032), 6 (1417548641880506368), 9 (1417548642062958592), 14 (1417548642071347201), 15 (1417548642077638657), 18 (1417548642089172994), 20 (1417548642102804480), 21 (1417548642435203072), 22 (1417548642439397377), 23 (1417548642444640257)]} 0 591
[junit4:junit4]   2> 16071 T103 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[65 (1417548642064007168), 69 (1417548642075541504), 71 (1417548642080784384), 73 (1417548642088124417), 76 (1417548642091270144), 79 (1417548642101755904), 80 (1417548642103853056), 82 (1417548642108047360), 83 (1417548642436251648), 84 (1417548642460368897)]} 0 403
[junit4:junit4]   2> 16082 T81 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[40 (1417548641880506368), 41 (1417548642071347200), 46 (1417548642084978688), 48 (1417548642092318720), 51 (1417548642096513025), 52 (1417548642106998784), 53 (1417548642432057344), 55 (1417548642460368896), 59 (1417548642462466048), 60 (1417548642486583296)]} 0 590
[junit4:junit4]   2> 16085 T51 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[24 (1417548641872117760), 27 (1417548642060861440), 28 (1417548642075541504), 34 (1417548642088124416), 38 (1417548642091270144), 39 (1417548642105950208), 41 (1417548642440445952), 43 (1417548642444640258), 44 (1417548642486583296), 46 (1417548642491826176)]} 0 601
[junit4:junit4]   2> 16098 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[59 (1417548641877360640), 61 (1417548642058764288), 62 (1417548642068201472), 63 (1417548642071347200), 64 (1417548642076590080), 65 (1417548642089172992), 69 (1417548642433105921), 73 (1417548642485534722), 74 (1417548642500214785), 75 (1417548642504409088)]} 0 609
[junit4:junit4]   2> 16100 T80 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[26 (1417548641872117760), 27 (1417548642067152897), 29 (1417548642088124416), 31 (1417548642096513024), 33 (1417548642427863040), 35 (1417548642438348800), 36 (1417548642446737408), 37 (1417548642451980288), 38 (1417548642490777602), 39 (1417548642500214784)]} 0 616
[junit4:junit4]   2> 16101 T67 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[49 (1417548641879457793), 52 (1417548642071347200), 55 (1417548642078687232), 56 (1417548642086027264), 59 (1417548642091270145), 60 (1417548642096513025), 63 (1417548642103853056), 65 (1417548642431008768), 66 (1417548642441494528), 69 (1417548642443591680)]} 0 609
[junit4:junit4]   2> 16107 T118 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[116 (1417548642097561600), 117 (1417548642437300225), 118 (1417548642485534721), 119 (1417548642491826178), 123 (1417548642499166208), 124 (1417548642501263361), 125 (1417548642503360512), 126 (1417548642505457665), 128 (1417548642519089152), 131 (1417548642521186304)]} 0 411
[junit4:junit4]   2> 16107 T116 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[113 (1417548642094415872), 115 (1417548642449883136), 116 (1417548642454077440), 117 (1417548642463514624), 119 (1417548642488680448), 120 (1417548642490777601), 122 (1417548642501263360), 124 (1417548642503360512), 125 (1417548642506506241), 126 (1417548642516992000)]} 0 411
[junit4:junit4]   2> 16110 T53 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[50 (1417548641877360640), 51 (1417548642069250048), 53 (1417548642076590080), 54 (1417548642082881537), 55 (1417548642092318721), 57 (1417548642097561600), 59 (1417548642431008768), 60 (1417548642448834560), 62 (1417548642511749120), 64 (1417548642514894848)]} 0 621
[junit4:junit4]   2> 16112 T66 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[26 (1417548641877360640), 27 (1417548641908817920), 28 (1417548642069250048), 33 (1417548642075541504), 36 (1417548642082881537), 37 (1417548642094415872), 40 (1417548642427863040), 43 (1417548642438348800), 45 (1417548642485534720), 48 (1417548642514894848)]} 0 622
[junit4:junit4]   2> 16115 T113 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[97 (1417548642083930112), 98 (1417548642096513024), 100 (1417548642431008768), 103 (1417548642489729024), 105 (1417548642491826177), 111 (1417548642501263360), 112 (1417548642509651969), 113 (1417548642514894849), 114 (1417548642519089153), 115 (1417548642522234881)]} 0 429
[junit4:junit4]   2> 16116 T115 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[98 (1417548642089172992), 102 (1417548642094415872), 103 (1417548642105950209), 106 (1417548642440445953), 108 (1417548642497069057), 111 (1417548642499166209), 112 (1417548642508603393), 115 (1417548642515943424), 116 (1417548642523283456), 117 (1417548642527477760)]} 0 424
[junit4:junit4]   2> 16117 T125 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[136 (1417548642435203072), 137 (1417548642442543104), 138 (1417548642445688832), 140 (1417548642448834560), 142 (1417548642451980288), 143 (1417548642455126016), 144 (1417548642460368896), 145 (1417548642465611776), 146 (1417548642468757504), 147 (1417548642527477760)]} 0 95
[junit4:junit4]   2> 16123 T102 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[77 (1417548642067152896), 79 (1417548642073444352), 80 (1417548642080784384), 81 (1417548642091270145), 82 (1417548642435203072), 83 (1417548642485534723), 84 (1417548642491826176), 89 (1417548642502311937), 90 (1417548642505457666), 96 (1417548642514894848)]} 0 452
[junit4:junit4]   2> 16136 T81 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[212 (1417548642520137728), 213 (1417548642523283456), 216 (1417548642525380608), 218 (1417548642527477761), 222 (1417548642532720641), 223 (1417548642539012096), 224 (1417548642541109249), 226 (1417548642548449280), 227 (1417548642550546432), 228 (1417548642552643584)]} 0 34
[junit4:junit4]   2> 16137 T127 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[118 (1417548642432057344), 120 (1417548642439397376), 126 (1417548642487631872), 127 (1417548642499166208), 130 (1417548642501263361), 131 (1417548642503360513), 132 (1417548642506506240), 134 (1417548642508603392), 136 (1417548642527477761), 137 (1417548642551595009)]} 0 119
[junit4:junit4]   2> 16138 T129 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[138 (1417548642459320320), 139 (1417548642497069058), 141 (1417548642503360512), 142 (1417548642512797696), 143 (1417548642519089152), 144 (1417548642522234880), 150 (1417548642524332032), 152 (1417548642531672064), 157 (1417548642549497856), 160 (1417548642553692161)]} 0 96
[junit4:junit4]   2> 16140 T133 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[155 (1417548642449883136), 158 (1417548642490777600), 159 (1417548642498117632), 160 (1417548642502311936), 161 (1417548642507554816), 162 (1417548642532720640), 164 (1417548642540060672), 165 (1417548642544254976), 166 (1417548642550546432), 168 (1417548642552643585)]} 0 104
[junit4:junit4]   2> 16144 T111 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[88 (1417548642082881536), 90 (1417548642095464448), 93 (1417548642434154496), 95 (1417548642484486145), 101 (1417548642491826176), 102 (1417548642502311936), 103 (1417548642504409089), 104 (1417548642527477762), 109 (1417548642541109248), 110 (1417548642543206400)]} 0 459
[junit4:junit4]   2> 16146 T148 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[152 (1417548642465611776), 153 (1417548642491826177), 154 (1417548642506506240), 158 (1417548642510700544), 159 (1417548642532720642), 162 (1417548642540060672), 171 (1417548642544254976), 172 (1417548642555789312), 173 (1417548642558935040), 174 (1417548642561032193)]} 0 95
[junit4:junit4]   2> 16147 T142 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[196 (1417548642501263360), 197 (1417548642537963521), 199 (1417548642542157825), 200 (1417548642548449280), 205 (1417548642551595008), 209 (1417548642553692160), 212 (1417548642555789312), 213 (1417548642557886465), 214 (1417548642562080768), 216 (1417548642564177921)]} 0 63
[junit4:junit4]   2> 16148 T143 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[170 (1417548642488680448), 171 (1417548642504409090), 172 (1417548642509651968), 173 (1417548642513846273), 175 (1417548642522234880), 176 (1417548642526429184), 179 (1417548642541109248), 180 (1417548642555789312), 183 (1417548642559983617), 185 (1417548642563129344)]} 0 76
[junit4:junit4]   2> 16154 T112 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[100 (1417548642088124416), 101 (1417548642097561600), 103 (1417548642427863041), 105 (1417548642431008769), 106 (1417548642439397376), 107 (1417548642441494529), 110 (1417548642456174592), 111 (1417548642524332034), 114 (1417548642529574912), 116 (1417548642540060672)]} 0 463
[junit4:junit4]   2> 16155 T158 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[270 (1417548642514894850), 271 (1417548642532720641), 272 (1417548642540060673), 274 (1417548642548449282), 275 (1417548642552643584), 276 (1417548642555789313), 280 (1417548642558935041), 283 (1417548642563129345), 284 (1417548642567323650), 285 (1417548642570469378)]} 0 57
[junit4:junit4]   2> 16155 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[210 (1417548642505457664), 211 (1417548642506506240), 212 (1417548642512797696), 214 (1417548642520137728), 216 (1417548642523283456), 224 (1417548642525380608), 227 (1417548642527477760), 228 (1417548642535866368), 229 (1417548642544254977), 230 (1417548642561032192)]} 0 67
[junit4:junit4]   2> 16157 T128 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[164 (1417548642485534720), 172 (1417548642504409088), 174 (1417548642515943425), 175 (1417548642519089153), 177 (1417548642533769216), 179 (1417548642555789313), 180 (1417548642558935043), 181 (1417548642564177920), 182 (1417548642569420800), 185 (1417548642571517953)]} 0 88
[junit4:junit4]   2> 16162 T103 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[266 (1417548642535866368), 267 (1417548642537963520), 269 (1417548642559983616), 270 (1417548642564177920), 271 (1417548642566275072), 272 (1417548642571517953), 273 (1417548642573615104), 275 (1417548642576760832), 279 (1417548642578857984), 284 (1417548642580955136)]} 0 45
[junit4:junit4]   2> 16164 T119 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[132 (1417548642437300224), 134 (1417548642485534720), 135 (1417548642500214784), 139 (1417548642504409089), 140 (1417548642508603392), 141 (1417548642511749120), 145 (1417548642516992000), 151 (1417548642534817792), 152 (1417548642567323648), 154 (1417548642573615104)]} 0 140
[junit4:junit4]   2> 16167 T121 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[117 (1417548642100707329), 119 (1417548642439397377), 121 (1417548642443591681), 122 (1417548642521186304), 123 (1417548642525380608), 125 (1417548642531672064), 130 (1417548642541109248), 131 (1417548642567323648), 132 (1417548642570469376), 134 (1417548642579906560)]} 0 465
[junit4:junit4]   2> 16171 T65 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[218 (1417548642509651968), 219 (1417548642521186305), 220 (1417548642524332032), 222 (1417548642527477762), 223 (1417548642533769216), 224 (1417548642549497857), 231 (1417548642554740736), 232 (1417548642557886464), 233 (1417548642580955136), 237 (1417548642586198016)]} 0 78
[junit4:junit4]   2> 16171 T116 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[247 (1417548642531672064), 249 (1417548642562080768), 250 (1417548642572566529), 251 (1417548642575712256), 252 (1417548642577809408), 254 (1417548642578857986), 259 (1417548642580955137), 260 (1417548642584100865), 262 (1417548642587246592), 263 (1417548642590392320)]} 0 57
[junit4:junit4]   2> 16171 T144 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[189 (1417548642513846272), 194 (1417548642534817793), 197 (1417548642548449281), 198 (1417548642554740736), 199 (1417548642558935040), 205 (1417548642567323649), 206 (1417548642572566528), 207 (1417548642575712256), 208 (1417548642580955136), 209 (1417548642587246592)]} 0 75
[junit4:junit4]   2> 16174 T146 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[221 (1417548642559983616), 224 (1417548642565226498), 225 (1417548642572566528), 226 (1417548642575712256), 228 (1417548642577809408), 229 (1417548642579906560), 230 (1417548642583052289), 233 (1417548642586198019), 234 (1417548642588295168), 236 (1417548642591440896)]} 0 50
[junit4:junit4]   2> 16171 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[232 (1417548642533769216), 236 (1417548642536914944), 238 (1417548642548449280), 240 (1417548642551595008), 241 (1417548642555789314), 242 (1417548642559983618), 243 (1417548642565226496), 244 (1417548642568372224), 245 (1417548642570469376), 247 (1417548642589343744)]} 0 66
[junit4:junit4]   2> 16174 T105 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[238 (1417548642542157824), 239 (1417548642552643584), 245 (1417548642558935042), 246 (1417548642565226497), 247 (1417548642568372224), 249 (1417548642573615104), 253 (1417548642583052288), 254 (1417548642586198018), 255 (1417548642589343744), 256 (1417548642592489472)]} 0 50
[junit4:junit4]   2> 16177 T98 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[257 (1417548642531672065), 260 (1417548642548449280), 262 (1417548642566275072), 264 (1417548642571517952), 268 (1417548642579906561), 274 (1417548642584100864), 276 (1417548642587246592), 279 (1417548642590392320), 280 (1417548642592489473), 281 (1417548642594586624)]} 0 63
[junit4:junit4]   2> 16173 T51 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[282 (1417548642541109248), 283 (1417548642551595008), 285 (1417548642555789312), 288 (1417548642558935041), 289 (1417548642565226496), 291 (1417548642569420801), 293 (1417548642573615105), 295 (1417548642582003712), 297 (1417548642586198017), 298 (1417548642589343746)]} 0 51
[junit4:junit4]   2> 16178 T66 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[276 (1417548642532720640), 278 (1417548642542157824), 279 (1417548642549497856), 281 (1417548642551595009), 285 (1417548642572566528), 288 (1417548642577809408), 289 (1417548642584100864), 290 (1417548642588295168), 291 (1417548642593538048), 293 (1417548642595635200)]} 0 64
[junit4:junit4]   2> 16181 T52 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[205 (1417548642542157825), 206 (1417548642550546432), 209 (1417548642553692160), 210 (1417548642580955136), 211 (1417548642586198016), 212 (1417548642589343745), 214 (1417548642593538048), 217 (1417548642595635200), 218 (1417548642598780928), 219 (1417548642599829504)]} 0 57
[junit4:junit4]   2> 16181 T80 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[231 (1417548642533769216), 232 (1417548642561032192), 233 (1417548642569420800), 234 (1417548642576760833), 235 (1417548642582003712), 236 (1417548642584100864), 237 (1417548642586198017), 238 (1417548642588295168), 239 (1417548642593538048), 241 (1417548642595635200)]} 0 73
[junit4:junit4]   2> 16181 T67 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[240 (1417548642521186306), 242 (1417548642524332033), 243 (1417548642541109249), 245 (1417548642561032193), 247 (1417548642574663680), 250 (1417548642587246592), 253 (1417548642590392320), 254 (1417548642593538049), 255 (1417548642596683776), 256 (1417548642598780928)]} 0 77
[junit4:junit4]   2> 16181 T118 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[286 (1417548642571517952), 291 (1417548642576760832), 292 (1417548642580955137), 293 (1417548642586198016), 294 (1417548642589343745), 295 (1417548642591440896), 296 (1417548642593538048), 298 (1417548642595635200), 306 (1417548642597732352), 307 (1417548642600878080)]} 0 29
[junit4:junit4]   2> 16182 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[249 (1417548642513846274), 253 (1417548642537963520), 256 (1417548642559983616), 260 (1417548642564177920), 262 (1417548642570469377), 263 (1417548642578857984), 265 (1417548642584100864), 266 (1417548642588295168), 268 (1417548642594586624), 269 (1417548642598780928)]} 0 85
[junit4:junit4]   2> 16187 T111 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[364 (1417548642574663680), 367 (1417548642578857985), 369 (1417548642582003713), 371 (1417548642586198016), 372 (1417548642590392321), 376 (1417548642599829504), 377 (1417548642602975232), 378 (1417548642604023808), 379 (1417548642605072384), 383 (1417548642606120960)]} 0 32
[junit4:junit4]   2> 16502 T78 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[128 (1417548642435203072), 133 (1417548642441494528), 135 (1417548642484486144), 136 (1417548642499166208), 139 (1417548642504409088), 141 (1417548642507554816), 143 (1417548642510700545), 144 (1417548642513846272), 145 (1417548642518040576), 148 (1417548642533769217)]} 0 480
[junit4:junit4]   2> 16502 T104 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[71 (1417548642072395776), 72 (1417548642082881536), 74 (1417548642089172992), 76 (1417548642094415873), 80 (1417548642100707328), 83 (1417548642436251648), 90 (1417548642487631872), 91 (1417548642523283457), 94 (1417548642525380609), 95 (1417548642530623489)]} 0 827
[junit4:junit4]   2> 16503 T145 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[187 (1417548642497069056), 189 (1417548642501263360), 190 (1417548642504409089), 191 (1417548642510700544), 193 (1417548642514894849), 197 (1417548642516992000), 199 (1417548642548449281), 200 (1417548642558935040), 202 (1417548642561032192), 204 (1417548642938519552)]} 0 431
[junit4:junit4]   2> 16504 T134 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[149 (1417548642459320320), 150 (1417548642518040576), 151 (1417548642520137728), 153 (1417548642523283456), 156 (1417548642527477761), 157 (1417548642533769217), 158 (1417548642561032192), 159 (1417548642570469377), 160 (1417548642576760833), 165 (1417548642583052289)]} 0 462
[junit4:junit4]   2> 16505 T161 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[257 (1417548642530623488), 260 (1417548642533769218), 262 (1417548642559983616), 263 (1417548642564177920), 265 (1417548642568372224), 266 (1417548642576760832), 269 (1417548642578857984), 272 (1417548642583052288), 274 (1417548642938519552), 275 (1417548642939568129)]} 0 393
[junit4:junit4]   2> 16505 T149 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[176 (1417548642490777600), 178 (1417548642507554817), 179 (1417548642527477763), 180 (1417548642532720640), 183 (1417548642537963521), 185 (1417548642540060673), 187 (1417548642559983617), 191 (1417548642564177921), 192 (1417548642566275073), 194 (1417548642571517952)]} 0 430
[junit4:junit4]   2> 16511 T124 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[196 (1417548642488680449), 197 (1417548642515943424), 198 (1417548642519089152), 200 (1417548642521186304), 201 (1417548642527477760), 204 (1417548642531672065), 205 (1417548642567323648), 206 (1417548642572566528), 207 (1417548642943762432), 209 (1417548642945859584)]} 0 439
[junit4:junit4]   2> 16517 T53 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[341 (1417548642938519553), 343 (1417548642940616704), 345 (1417548642941665280), 346 (1417548642942713856), 348 (1417548642944811008), 349 (1417548642945859584), 350 (1417548642946908161), 352 (1417548642950053888), 354 (1417548642951102464), 355 (1417548642953199616)]} 0 15
[junit4:junit4]   2> 16519 T133 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[328 (1417548642942713856), 329 (1417548642943762433), 332 (1417548642944811009), 333 (1417548642945859585), 336 (1417548642947956737), 339 (1417548642949005313), 343 (1417548642950053889), 344 (1417548642951102465), 345 (1417548642953199616), 346 (1417548642954248193)]} 0 14
[junit4:junit4]   2> 16520 T113 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[348 (1417548642941665280), 350 (1417548642943762432), 351 (1417548642944811008), 352 (1417548642945859584), 353 (1417548642947956736), 360 (1417548642949005312), 361 (1417548642950053888), 362 (1417548642951102464), 366 (1417548642953199617), 368 (1417548642954248194)]} 0 15
[junit4:junit4]   2> 16520 T141 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[167 (1417548642489729024), 172 (1417548642514894849), 173 (1417548642537963520), 181 (1417548642940616704), 183 (1417548642943762432), 187 (1417548642945859584), 188 (1417548642947956737), 191 (1417548642949005314), 193 (1417548642951102466), 195 (1417548642954248192)]} 0 447
[junit4:junit4]   2> 16523 T81 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[287 (1417548642939568128), 289 (1417548642941665280), 291 (1417548642943762433), 296 (1417548642945859585), 299 (1417548642947956736), 300 (1417548642949005312), 301 (1417548642951102464), 304 (1417548642953199616), 305 (1417548642956345344), 306 (1417548642958442496)]} 0 20
[junit4:junit4]   2> 16525 T125 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[343 (1417548642943762433), 344 (1417548642944811009), 346 (1417548642945859585), 348 (1417548642947956738), 349 (1417548642949005313), 350 (1417548642950053890), 352 (1417548642952151040), 353 (1417548642953199618), 356 (1417548642954248193), 364 (1417548642957393920)]} 0 18
[junit4:junit4]   2> 16528 T115 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[356 (1417548642941665281), 357 (1417548642943762432), 359 (1417548642944811009), 363 (1417548642945859585), 366 (1417548642949005312), 367 (1417548642950053889), 368 (1417548642952151040), 371 (1417548642956345344), 373 (1417548642958442496), 376 (1417548642962636800)]} 0 23
[junit4:junit4]   2> 16530 T112 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[320 (1417548642944811008), 322 (1417548642946908161), 326 (1417548642949005312), 328 (1417548642951102464), 329 (1417548642953199617), 332 (1417548642955296769), 333 (1417548642957393922), 336 (1417548642960539649), 339 (1417548642963685377), 341 (1417548642965782528)]} 0 22
[junit4:junit4]   2> 16531 T159 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[295 (1417548642939568128), 296 (1417548642940616705), 298 (1417548642942713856), 301 (1417548642947956736), 302 (1417548642950053889), 304 (1417548642953199616), 308 (1417548642956345344), 315 (1417548642959491073), 317 (1417548642963685376), 318 (1417548642967879681)]} 0 29
[junit4:junit4]   2> 16536 T118 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[403 (1417548642957393921), 405 (1417548642960539648), 407 (1417548642962636800), 411 (1417548642963685377), 412 (1417548642965782528), 413 (1417548642966831104), 414 (1417548642967879682), 417 (1417548642969976833), 418 (1417548642971025410), 419 (1417548642973122560)]} 0 16
[junit4:junit4]   2> 16538 T158 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[369 (1417548642954248192), 372 (1417548642955296768), 373 (1417548642959491072), 374 (1417548642961588225), 378 (1417548642963685376), 379 (1417548642965782530), 382 (1417548642967879681), 384 (1417548642969976834), 385 (1417548642972073985), 387 (1417548642974171136)]} 0 22
[junit4:junit4]   2> 16540 T128 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[300 (1417548642944811010), 302 (1417548642946908160), 303 (1417548642949005313), 304 (1417548642951102465), 306 (1417548642954248192), 307 (1417548642960539649), 309 (1417548642963685376), 311 (1417548642967879682), 315 (1417548642971025409), 319 (1417548642976268288)]} 0 32
[junit4:junit4]   2> 16592 T65 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[389 (1417548642946908160), 390 (1417548642951102465), 393 (1417548642955296768), 399 (1417548642957393921), 400 (1417548642960539648), 401 (1417548642965782529), 402 (1417548642968928256), 404 (1417548642973122561), 405 (1417548642975219713), 406 (1417548642977316865)]} 0 83
[junit4:junit4]   2> 16593 T146 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[392 (1417548642957393920), 393 (1417548642960539648), 395 (1417548642961588224), 396 (1417548642965782528), 399 (1417548642967879681), 401 (1417548642968928256), 403 (1417548642973122560), 405 (1417548642976268289), 407 (1417548642977316864), 408 (1417548642979414016)]} 0 74
[junit4:junit4]   2> 16595 T66 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[365 (1417548642950053888), 369 (1417548642952151041), 372 (1417548642955296770), 373 (1417548642964733952), 374 (1417548642967879680), 375 (1417548642971025409), 376 (1417548642975219712), 377 (1417548642977316864), 381 (1417548642978365440), 388 (1417548643033939968)]} 0 82
[junit4:junit4]   2> 16596 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[308 (1417548642962636801), 309 (1417548642963685378), 313 (1417548642965782531), 314 (1417548642967879680), 315 (1417548642971025409), 319 (1417548642973122561), 321 (1417548642975219712), 325 (1417548642976268289), 326 (1417548642979414017), 327 (1417548643033939970)]} 0 71
[junit4:junit4]   2> 16599 T144 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[494 (1417548642968928257), 496 (1417548642972073984), 497 (1417548642973122562), 498 (1417548642976268288), 501 (1417548642977316865), 503 (1417548643032891392), 506 (1417548643033939969), 507 (1417548643036037121), 508 (1417548643038134272), 509 (1417548643039182849)]} 0 68
[junit4:junit4]   2> 16599 T119 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[450 (1417548642956345344), 451 (1417548642958442496), 453 (1417548642961588224), 454 (1417548642964733952), 456 (1417548642969976832), 460 (1417548642972073986), 461 (1417548642975219713), 466 (1417548642979414016), 470 (1417548643034988544), 476 (1417548643037085696)]} 0 81
[junit4:junit4]   2> 16599 T116 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[427 (1417548642955296768), 428 (1417548642957393920), 429 (1417548642959491073), 430 (1417548642964733952), 433 (1417548642965782528), 437 (1417548642969976832), 439 (1417548642972073984), 441 (1417548642974171138), 444 (1417548642976268288), 445 (1417548642979414016)]} 0 81
[junit4:junit4]   2> 16609 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[420 (1417548642957393920), 421 (1417548642960539649), 422 (1417548642965782529), 423 (1417548642968928256), 426 (1417548642971025408), 429 (1417548642974171137), 434 (1417548642977316864), 435 (1417548643033939968), 439 (1417548643036037120), 443 (1417548643038134273)]} 0 89
[junit4:junit4]   2> 16618 T98 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[450 (1417548642960539650), 451 (1417548642963685377), 452 (1417548642967879680), 454 (1417548642971025408), 455 (1417548642975219712), 456 (1417548642978365440), 458 (1417548643042328576), 461 (1417548643044425728), 463 (1417548643045474304), 465 (1417548643057008642)]} 0 94
[junit4:junit4]   2> 16623 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[388 (1417548643040231424), 390 (1417548643045474305), 392 (1417548643050717185), 394 (1417548643052814336), 395 (1417548643053862912), 396 (1417548643054911489), 398 (1417548643057008641), 399 (1417548643059105792), 400 (1417548643061202944), 402 (1417548643063300096)]} 0 25
[junit4:junit4]   2> 16623 T80 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[406 (1417548642959491072), 410 (1417548642967879681), 412 (1417548642969976833), 414 (1417548642974171136), 416 (1417548642975219712), 417 (1417548642977316865), 419 (1417548642979414017), 421 (1417548643043377152), 424 (1417548643053862912), 425 (1417548643062251521)]} 0 101
[junit4:junit4]   2> 16633 T51 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[509 (1417548643034988544), 510 (1417548643036037120), 511 (1417548643038134272), 516 (1417548643039182849), 517 (1417548643040231424), 518 (1417548643041280001), 521 (1417548643043377152), 522 (1417548643044425729), 523 (1417548643059105792), 526 (1417548643066445824)]} 0 39
[junit4:junit4]   2> 16637 T104 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[453 (1417548642956345345), 454 (1417548642959491072), 461 (1417548642961588224), 462 (1417548642964733953), 463 (1417548642967879682), 468 (1417548642971025408), 469 (1417548642973122560), 470 (1417548643050717184), 472 (1417548643072737280), 477 (1417548643073785857)]} 0 116
[junit4:junit4]   2> 16647 T102 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[510 (1417548643037085697), 515 (1417548643039182848), 520 (1417548643040231425), 521 (1417548643041280001), 523 (1417548643043377152), 527 (1417548643044425729), 529 (1417548643050717184), 530 (1417548643052814337), 534 (1417548643054911488), 536 (1417548643059105793)]} 0 51
[junit4:junit4]   2> 16647 T158 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[583 (1417548643060154369), 586 (1417548643065397248), 588 (1417548643072737280), 590 (1417548643073785857), 591 (1417548643076931584), 593 (1417548643084271616), 594 (1417548643085320193), 596 (1417548643086368769), 597 (1417548643088465920), 600 (1417548643089514496)]} 0 33
[junit4:junit4]   2> 16648 T143 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[478 (1417548643041280000), 480 (1417548643047571456), 481 (1417548643051765760), 482 (1417548643052814338), 483 (1417548643053862914), 486 (1417548643055960064), 487 (1417548643057008642), 488 (1417548643058057216), 489 (1417548643061202945), 493 (1417548643084271618)]} 0 48
[junit4:junit4]   2> 16649 T113 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[537 (1417548643044425728), 541 (1417548643045474304), 542 (1417548643057008640), 545 (1417548643059105794), 548 (1417548643084271617), 549 (1417548643085320194), 551 (1417548643087417344), 554 (1417548643088465922), 558 (1417548643089514498), 562 (1417548643091611648)]} 0 47
[junit4:junit4]   2> 16652 T65 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[421 (1417548643036037120), 425 (1417548643038134272), 426 (1417548643042328576), 428 (1417548643044425729), 429 (1417548643058057216), 431 (1417548643073785859), 432 (1417548643082174465), 433 (1417548643085320194), 435 (1417548643087417346), 436 (1417548643089514498)]} 0 58
[junit4:junit4]   2> 16653 T160 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[407 (1417548643034988544), 409 (1417548643037085696), 410 (1417548643039182849), 411 (1417548643049668608), 412 (1417548643054911489), 414 (1417548643066445824), 416 (1417548643076931584), 418 (1417548643083223040), 419 (1417548643092660224), 420 (1417548643095805952)]} 0 59
[junit4:junit4]   2> 16653 T118 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[564 (1417548643049668608), 569 (1417548643053862913), 571 (1417548643055960065), 573 (1417548643060154368), 574 (1417548643066445824), 575 (1417548643073785856), 576 (1417548643076931585), 577 (1417548643092660224), 581 (1417548643093708800), 582 (1417548643095805952)]} 0 45
[junit4:junit4]   2> 16656 T124 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[446 (1417548643040231425), 449 (1417548643047571459), 450 (1417548643052814339), 453 (1417548643055960064), 455 (1417548643065397250), 456 (1417548643077980160), 457 (1417548643082174464), 459 (1417548643083223041), 460 (1417548643093708802), 461 (1417548643094757377)]} 0 57
[junit4:junit4]   2> 16661 T98 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[591 (1417548643064348672), 593 (1417548643081125888), 594 (1417548643083223041), 598 (1417548643085320194), 604 (1417548643087417347), 605 (1417548643090563072), 606 (1417548643091611649), 608 (1417548643095805954), 609 (1417548643097903104), 610 (1417548643101048832)]} 0 39
[junit4:junit4]   2> 16658 T129 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[320 (1417548643041280003), 323 (1417548643051765761), 326 (1417548643077980160), 328 (1417548643085320192), 331 (1417548643087417344), 332 (1417548643089514496), 335 (1417548643090563074), 336 (1417548643095805952), 339 (1417548643097903105), 340 (1417548643100000256)]} 0 58
[junit4:junit4]   2> 16663 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[603 (1417548643065397249), 604 (1417548643085320192), 605 (1417548643086368768), 606 (1417548643088465921), 608 (1417548643089514497), 609 (1417548643094757376), 610 (1417548643098951680), 611 (1417548643100000256), 613 (1417548643102097408), 617 (1417548643106291712)]} 0 40
[junit4:junit4]   2> 16664 T105 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[468 (1417548643057008641), 470 (1417548643079028738), 472 (1417548643086368769), 473 (1417548643088465921), 477 (1417548643092660225), 480 (1417548643095805953), 483 (1417548643097903106), 484 (1417548643100000257), 485 (1417548643105243136), 487 (1417548643107340288)]} 0 49
[junit4:junit4]   2> 16666 T149 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[385 (1417548643040231424), 388 (1417548643057008640), 391 (1417548643063300096), 392 (1417548643074834432), 396 (1417548643080077312), 398 (1417548643084271617), 399 (1417548643086368770), 401 (1417548643093708800), 404 (1417548643094757376), 405 (1417548643098951680)]} 0 67
[junit4:junit4]   2> 16666 T52 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[377 (1417548643039182848), 378 (1417548643047571458), 379 (1417548643051765762), 382 (1417548643053862912), 383 (1417548643054911488), 384 (1417548643079028736), 386 (1417548643081125889), 387 (1417548643083223040), 388 (1417548643106291712), 389 (1417548643108388864)]} 0 69
[junit4:junit4]   2> 16667 T165 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[511 (1417548643052814337), 512 (1417548643063300097), 513 (1417548643072737281), 519 (1417548643084271618), 522 (1417548643086368769), 523 (1417548643088465920), 524 (1417548643093708803), 528 (1417548643097903106), 531 (1417548643106291712), 532 (1417548643108388864)]} 0 56
[junit4:junit4]   2> 16657 T81 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[308 (1417548643047571456), 309 (1417548643053862913), 311 (1417548643061202944), 316 (1417548643072737282), 317 (1417548643074834433), 320 (1417548643081125888), 325 (1417548643082174466), 328 (1417548643086368768), 332 (1417548643095805952), 335 (1417548643096854528)]} 0 52
[junit4:junit4]   2> 16678 T128 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[554 (1417548643049668608), 555 (1417548643055960064), 556 (1417548643058057218), 559 (1417548643075883008), 565 (1417548643085320193), 566 (1417548643090563073), 567 (1417548643096854528), 569 (1417548643101048833), 570 (1417548643110486016), 574 (1417548643111534592)]} 0 70
[junit4:junit4]   2> 16679 T148 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[342 (1417548643034988544), 343 (1417548643051765760), 344 (1417548643054911489), 347 (1417548643062251520), 348 (1417548643064348672), 349 (1417548643080077316), 350 (1417548643082174468), 352 (1417548643093708801), 361 (1417548643097903104), 363 (1417548643102097409)]} 0 85
[junit4:junit4]   2> 16679 T121 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[540 (1417548643040231424), 541 (1417548643051765760), 543 (1417548643061202945), 544 (1417548643084271617), 546 (1417548643086368768), 547 (1417548643089514497), 548 (1417548643100000259), 552 (1417548643108388864), 555 (1417548643110486016), 557 (1417548643123068928)]} 0 80
[junit4:junit4]   2> 16657 T111 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[462 (1417548642960539648), 463 (1417548642962636800), 465 (1417548642967879680), 466 (1417548642972073985), 468 (1417548642974171137), 469 (1417548642977316864), 470 (1417548643032891392), 471 (1417548643050717185), 473 (1417548643084271616), 474 (1417548643093708804)]} 0 134
[junit4:junit4]   2> 16679 T159 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[608 (1417548643075883008), 609 (1417548643077980160), 611 (1417548643087417344), 612 (1417548643090563072), 614 (1417548643093708800), 617 (1417548643096854529), 618 (1417548643098951681), 619 (1417548643102097408), 625 (1417548643107340288), 626 (1417548643111534593)]} 0 46
[junit4:junit4]   2> 16668 T116 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[554 (1417548643050717184), 555 (1417548643058057217), 557 (1417548643060154368), 558 (1417548643065397249), 559 (1417548643074834434), 561 (1417548643080077313), 569 (1417548643082174467), 575 (1417548643088465921), 577 (1417548643105243136), 578 (1417548643110486017)]} 0 62
[junit4:junit4]   2> 16668 T53 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[432 (1417548643041280002), 433 (1417548643053862913), 436 (1417548643057008640), 437 (1417548643086368768), 439 (1417548643089514497), 440 (1417548643094757377), 443 (1417548643095805955), 445 (1417548643101048834), 446 (1417548643107340290), 449 (1417548643109437441)]} 0 68
[junit4:junit4]   2> 16667 T67 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[478 (1417548643047571456), 479 (1417548643053862912), 480 (1417548643073785858), 481 (1417548643087417345), 483 (1417548643089514496), 484 (1417548643092660225), 485 (1417548643097903106), 486 (1417548643106291712), 492 (1417548643109437440), 493 (1417548643110486017)]} 0 62
[junit4:junit4]   2> 16663 T141 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[437 (1417548643044425728), 440 (1417548643045474304), 442 (1417548643052814336), 443 (1417548643054911488), 445 (1417548643055960065), 447 (1417548643061202946), 449 (1417548643084271616), 450 (1417548643088465920), 451 (1417548643094757376), 452 (1417548643100000257)]} 0 60
[junit4:junit4]   2> 16662 T145 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[409 (1417548643039182850), 411 (1417548643041280000), 412 (1417548643047571456), 414 (1417548643051765760), 417 (1417548643055960065), 420 (1417548643072737280), 422 (1417548643087417345), 423 (1417548643091611648), 427 (1417548643094757376), 430 (1417548643102097408)]} 0 64
[junit4:junit4]   2> 16685 T164 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[475 (1417548643045474304), 476 (1417548643053862914), 477 (1417548643062251523), 479 (1417548643065397248), 480 (1417548643072737280), 482 (1417548643075883008), 486 (1417548643081125889), 488 (1417548643107340289), 489 (1417548643109437440), 491 (1417548643126214656)]} 0 82
[junit4:junit4]   2> 16686 T78 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[492 (1417548643047571458), 496 (1417548643052814338), 497 (1417548643053862915), 498 (1417548643080077315), 499 (1417548643083223040), 503 (1417548643097903105), 504 (1417548643102097408), 505 (1417548643107340290), 507 (1417548643126214657), 510 (1417548643129360385)]} 0 80
[junit4:junit4]   2> 16662 T125 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[559 (1417548643057008640), 560 (1417548643074834432), 562 (1417548643082174466), 571 (1417548643085320192), 573 (1417548643088465921), 577 (1417548643090563073), 578 (1417548643092660227), 580 (1417548643094757377), 581 (1417548643097903104), 584 (1417548643100000258)]} 0 54
[junit4:junit4]   2> 16687 T166 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[582 (1417548643066445824), 583 (1417548643073785857), 584 (1417548643080077314), 587 (1417548643092660224), 590 (1417548643095805953), 591 (1417548643107340288), 594 (1417548643125166081), 596 (1417548643127263232), 599 (1417548643129360384), 604 (1417548643131457536)]} 0 64
[junit4:junit4]   2> 16688 T146 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[575 (1417548643049668610), 576 (1417548643058057216), 577 (1417548643077980161), 579 (1417548643088465920), 583 (1417548643089514498), 584 (1417548643107340289), 585 (1417548643109437442), 586 (1417548643126214659), 589 (1417548643129360384), 590 (1417548643131457537)]} 0 80
[junit4:junit4]   2> 16662 T112 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[586 (1417548643063300096), 587 (1417548643085320193), 592 (1417548643086368769), 594 (1417548643091611648), 595 (1417548643093708801), 597 (1417548643096854528), 599 (1417548643097903105), 604 (1417548643100000256), 606 (1417548643101048832), 607 (1417548643105243136)]} 0 41
[junit4:junit4]   2> 16691 T115 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[489 (1417548643047571457), 490 (1417548643058057217), 491 (1417548643080077312), 493 (1417548643106291713), 494 (1417548643109437440), 495 (1417548643126214657), 499 (1417548643129360386), 501 (1417548643131457539), 506 (1417548643133554688), 507 (1417548643134603265)]} 0 86
[junit4:junit4]   2> 16682 T127 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[527 (1417548643049668609), 532 (1417548643079028737), 533 (1417548643087417346), 537 (1417548643092660224), 539 (1417548643096854529), 545 (1417548643098951680), 548 (1417548643108388865), 549 (1417548643110486018), 550 (1417548643124117504), 552 (1417548643126214656)]} 0 74
[junit4:junit4]   2> 16682 T134 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[497 (1417548643042328577), 499 (1417548643058057217), 501 (1417548643061202944), 502 (1417548643073785856), 504 (1417548643082174464), 505 (1417548643083223041), 508 (1417548643106291713), 511 (1417548643108388865), 512 (1417548643111534592), 514 (1417548643126214656)]} 0 141
[junit4:junit4]   2> 16682 T103 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[534 (1417548643047571457), 536 (1417548643052814336), 538 (1417548643054911488), 540 (1417548643058057216), 541 (1417548643062251522), 542 (1417548643073785856), 543 (1417548643076931584), 545 (1417548643082174465), 551 (1417548643110486016), 552 (1417548643112583168)]} 0 77
[junit4:junit4]   2> 16680 T66 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[515 (1417548643039182848), 516 (1417548643041280000), 519 (1417548643055960064), 523 (1417548643064348672), 531 (1417548643066445825), 532 (1417548643079028736), 536 (1417548643092660226), 537 (1417548643098951680), 538 (1417548643105243137), 539 (1417548643110486018)]} 0 83
[junit4:junit4]   2> 16697 T142 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[629 (1417548643129360384), 630 (1417548643131457536), 632 (1417548643131457537), 634 (1417548643132506112), 637 (1417548643133554688), 638 (1417548643134603264), 639 (1417548643136700416), 640 (1417548643138797568), 645 (1417548643139846144), 648 (1417548643141943296)]} 0 13
[junit4:junit4]   2> 16697 T129 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[612 (1417548643110486017), 613 (1417548643123068928), 617 (1417548643126214658), 618 (1417548643129360385), 619 (1417548643131457536), 625 (1417548643132506112), 628 (1417548643133554689), 629 (1417548643136700416), 630 (1417548643138797569), 635 (1417548643140894721)]} 0 32
[junit4:junit4]   2> 16704 T124 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[637 (1417548643125166080), 638 (1417548643129360386), 640 (1417548643131457537), 641 (1417548643133554688), 643 (1417548643136700416), 646 (1417548643138797568), 647 (1417548643140894720), 649 (1417548643142991872), 651 (1417548643146137600), 653 (1417548643148234753)]} 0 25
[junit4:junit4]   2> 16704 T111 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[655 (1417548643127263233), 656 (1417548643130408960), 658 (1417548643132506112), 659 (1417548643133554689), 661 (1417548643134603264), 662 (1417548643137748992), 663 (1417548643139846144), 665 (1417548643141943298), 667 (1417548643144040448), 668 (1417548643147186177)]} 0 22
[junit4:junit4]   2> 16706 T51 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[637 (1417548643131457538), 638 (1417548643132506113), 640 (1417548643134603264), 647 (1417548643135651840), 648 (1417548643137748992), 649 (1417548643138797570), 651 (1417548643140894720), 654 (1417548643145089024), 655 (1417548643149283328), 656 (1417548643150331905)]} 0 20
[junit4:junit4]   2> 16711 T105 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[678 (1417548643139846145), 681 (1417548643141943297), 683 (1417548643142991873), 686 (1417548643146137600), 687 (1417548643147186176), 690 (1417548643148234753), 691 (1417548643149283329), 695 (1417548643150331906), 696 (1417548643152429057), 698 (1417548643155574785)]} 0 17
[junit4:junit4]   2> 16711 T116 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[605 (1417548643135651840), 609 (1417548643138797569), 612 (1417548643141943296), 613 (1417548643145089024), 617 (1417548643147186176), 619 (1417548643148234754), 623 (1417548643149283329), 628 (1417548643151380481), 633 (1417548643153477633), 635 (1417548643155574786)]} 0 22
[junit4:junit4]   2> 16713 T119 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[639 (1417548643130408960), 640 (1417548643132506112), 646 (1417548643133554688), 647 (1417548643134603264), 649 (1417548643136700416), 651 (1417548643138797568), 653 (1417548643145089025), 654 (1417548643148234752), 655 (1417548643150331904), 657 (1417548643153477632)]} 0 28
[junit4:junit4]   2> 16717 T98 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[657 (1417548643136700417), 658 (1417548643138797568), 659 (1417548643139846144), 660 (1417548643141943296), 664 (1417548643142991872), 668 (1417548643145089025), 670 (1417548643148234752), 671 (1417548643150331904), 672 (1417548643152429056), 677 (1417548643154526208)]} 0 26
[junit4:junit4]   2> 16719 T80 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[670 (1417548643141943297), 672 (1417548643148234752), 674 (1417548643149283328), 675 (1417548643150331904), 680 (1417548643153477634), 683 (1417548643155574785), 684 (1417548643156623360), 685 (1417548643157671936), 686 (1417548643160817664), 689 (1417548643162914817)]} 0 23
[junit4:junit4]   2> 16722 T161 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[652 (1417548643137748992), 655 (1417548643139846145), 656 (1417548643141943297), 658 (1417548643145089024), 659 (1417548643147186176), 662 (1417548643150331905), 663 (1417548643153477632), 665 (1417548643155574784), 667 (1417548643160817664), 669 (1417548643161866240)]} 0 29
[junit4:junit4]   2> 16722 T158 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[718 (1417548643152429057), 720 (1417548643154526208), 723 (1417548643155574784), 724 (1417548643157671936), 726 (1417548643158720512), 727 (1417548643160817664), 728 (1417548643161866240), 729 (1417548643162914817), 731 (1417548643163963393), 732 (1417548643165011970)]} 0 16
[junit4:junit4]   2> 16722 T133 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[620 (1417548643137748992), 621 (1417548643139846144), 622 (1417548643146137600), 626 (1417548643148234753), 627 (1417548643153477633), 628 (1417548643157671937), 631 (1417548643160817665), 632 (1417548643162914816), 635 (1417548643165011968), 636 (1417548643167109120)]} 0 31
[junit4:junit4]   2> 16725 T81 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[690 (1417548643148234755), 692 (1417548643151380480), 693 (1417548643153477632), 694 (1417548643155574784), 695 (1417548643160817665), 696 (1417548643162914818), 698 (1417548643165011968), 700 (1417548643167109120), 702 (1417548643168157697), 703 (1417548643170254849)]} 0 23
[junit4:junit4]   2> 16726 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[658 (1417548643147186176), 661 (1417548643152429056), 662 (1417548643156623360), 663 (1417548643158720513), 668 (1417548643160817667), 669 (1417548643163963392), 671 (1417548643166060544), 672 (1417548643168157696), 673 (1417548643169206273), 674 (1417548643172352000)]} 0 25
[junit4:junit4]   2> 16727 T104 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[691 (1417548643146137600), 693 (1417548643148234753), 694 (1417548643149283328), 695 (1417548643151380480), 700 (1417548643156623360), 702 (1417548643157671936), 708 (1417548643159769088), 709 (1417548643162914816), 710 (1417548643169206272), 712 (1417548643171303424)]} 0 26
[junit4:junit4]   2> 16727 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[697 (1417548643147186178), 700 (1417548643150331905), 701 (1417548643153477634), 705 (1417548643157671939), 706 (1417548643160817666), 707 (1417548643162914818), 708 (1417548643165011969), 711 (1417548643167109121), 712 (1417548643171303424), 716 (1417548643173400576)]} 0 26
[junit4:junit4]   2> 16729 T164 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[721 (1417548643159769089), 726 (1417548643160817666), 729 (1417548643162914816), 730 (1417548643163963392), 731 (1417548643167109121), 733 (1417548643168157696), 734 (1417548643169206272), 735 (1417548643170254848), 736 (1417548643172352000), 737 (1417548643174449152)]} 0 16
[junit4:junit4]   2> 16729 T144 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[675 (1417548643145089024), 678 (1417548643147186177), 681 (1417548643148234754), 682 (1417548643154526209), 684 (1417548643157671938), 685 (1417548643169206272), 688 (1417548643170254848), 691 (1417548643172352001), 693 (1417548643174449152), 694 (1417548643175497728)]} 0 31
[junit4:junit4]   2> 16730 T148 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[705 (1417548643152429056), 707 (1417548643154526208), 709 (1417548643156623361), 710 (1417548643159769088), 711 (1417548643160817667), 713 (1417548643163963393), 714 (1417548643169206273), 715 (1417548643171303424), 717 (1417548643173400576), 718 (1417548643175497728)]} 0 24
[junit4:junit4]   2> 16731 T52 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[699 (1417548643152429058), 704 (1417548643155574784), 705 (1417548643160817665), 706 (1417548643162914816), 707 (1417548643166060544), 708 (1417548643167109120), 709 (1417548643168157696), 710 (1417548643170254848), 711 (1417548643172352000), 712 (1417548643174449152)]} 0 25
[junit4:junit4]   2> 16739 T65 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[670 (1417548643148234752), 672 (1417548643150331904), 674 (1417548643155574785), 676 (1417548643162914817), 683 (1417548643163963393), 685 (1417548643165011968), 686 (1417548643172352000), 688 (1417548643174449152), 689 (1417548643176546304), 690 (1417548643180740608)]} 0 38
[junit4:junit4]   2> 16741 T128 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[714 (1417548643156623360), 720 (1417548643157671936), 721 (1417548643160817664), 722 (1417548643161866240), 725 (1417548643169206272), 726 (1417548643171303424), 727 (1417548643173400576), 728 (1417548643175497729), 733 (1417548643178643456), 734 (1417548643185983488)]} 0 31
[junit4:junit4]   2> 16743 T160 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[717 (1417548643160817665), 718 (1417548643163963392), 721 (1417548643166060544), 722 (1417548643168157696), 724 (1417548643170254848), 726 (1417548643175497728), 728 (1417548643178643456), 732 (1417548643182837761), 735 (1417548643188080640), 737 (1417548643190177792)]} 0 29
[junit4:junit4]   2> 16749 T145 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[736 (1417548643173400577), 740 (1417548643174449153), 742 (1417548643175497728), 743 (1417548643177594880), 744 (1417548643181789184), 745 (1417548643187032064), 748 (1417548643190177792), 751 (1417548643192274944), 754 (1417548643193323521), 755 (1417548643195420672)]} 0 23
[junit4:junit4]   2> 16754 T53 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[793 (1417548643189129216), 794 (1417548643190177793), 795 (1417548643191226369), 796 (1417548643193323520), 798 (1417548643194372096), 799 (1417548643195420673), 801 (1417548643197517824), 805 (1417548643198566401), 808 (1417548643199614976), 809 (1417548643200663553)]} 0 13
[junit4:junit4]   2> 16755 T121 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[742 (1417548643175497729), 743 (1417548643176546305), 747 (1417548643181789184), 748 (1417548643182837760), 749 (1417548643188080641), 750 (1417548643190177793), 751 (1417548643192274945), 752 (1417548643194372096), 753 (1417548643196469248), 754 (1417548643198566401)]} 0 27
[junit4:junit4]   2> 16757 T149 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[753 (1417548643185983489), 755 (1417548643188080641), 756 (1417548643190177793), 757 (1417548643192274944), 758 (1417548643193323520), 760 (1417548643195420672), 762 (1417548643197517824), 763 (1417548643199614977), 765 (1417548643201712128), 767 (1417548643203809280)]} 0 22
[junit4:junit4]   2> 16758 T143 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[733 (1417548643185983488), 736 (1417548643188080640), 738 (1417548643190177793), 739 (1417548643192274944), 740 (1417548643194372097), 745 (1417548643196469249), 750 (1417548643198566403), 754 (1417548643200663553), 764 (1417548643202760705), 765 (1417548643204857857)]} 0 20
[junit4:junit4]   2> 16760 T78 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[738 (1417548643181789184), 741 (1417548643185983488), 742 (1417548643187032064), 744 (1417548643188080642), 745 (1417548643191226368), 747 (1417548643194372096), 748 (1417548643196469248), 750 (1417548643199614976), 751 (1417548643201712129), 752 (1417548643205906433)]} 0 26
[junit4:junit4]   2> 16762 T113 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[766 (1417548643190177792), 771 (1417548643192274945), 772 (1417548643194372096), 773 (1417548643196469248), 778 (1417548643198566402), 779 (1417548643200663552), 781 (1417548643201712129), 782 (1417548643204857856), 784 (1417548643206955008), 785 (1417548643209052160)]} 0 20
[junit4:junit4]   2> 16765 T159 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[773 (1417548643188080642), 774 (1417548643190177794), 781 (1417548643192274944), 782 (1417548643193323520), 783 (1417548643195420672), 784 (1417548643197517824), 786 (1417548643200663552), 787 (1417548643202760704), 788 (1417548643204857856), 789 (1417548643212197888)]} 0 25
[junit4:junit4]   2> 16768 T102 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[786 (1417548643195420673), 788 (1417548643198566400), 790 (1417548643199614977), 791 (1417548643201712128), 794 (1417548643203809280), 795 (1417548643206955009), 797 (1417548643209052161), 798 (1417548643211149312), 800 (1417548643213246464), 802 (1417548643215343616)]} 0 21
[junit4:junit4]   2> 16768 T166 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[794 (1417548643191226369), 798 (1417548643193323521), 799 (1417548643196469249), 801 (1417548643199614978), 802 (1417548643201712130), 804 (1417548643204857856), 806 (1417548643206955008), 807 (14175486432101

[...truncated too long message...]

dThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene41: {timestamp=PostingsFormat(name=Direct), range_facet_l=PostingsFormat(name=Direct), id=Pulsing41(freqCutoff=2 minBlockSize=50 maxBlockSize=174), range_facet_sl=PostingsFormat(name=Lucene41WithOrds), range_facet_si=MockFixedIntBlock(blockSize=1544), multiDefault=MockFixedIntBlock(blockSize=1544), intDefault=PostingsFormat(name=Direct)}, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=es_GT, timezone=Pacific/Truk
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=5,free=77576888,total=312475648
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestFoldingMultitermQuery, IndexSchemaRuntimeFieldTest, DirectSolrSpellCheckerTest, RAMDirectoryFactoryTest, StatsComponentTest, TestSurroundQueryParser, RequiredFieldsTest, DocumentAnalysisRequestHandlerTest, BadIndexSchemaTest, TestSolrDeletionPolicy1, TestWriterPerf, TestCSVResponseWriter, DOMUtilTest, StatelessScriptUpdateProcessorFactoryTest, ZkCLITest, DebugComponentTest, LeaderElectionTest, TestReversedWildcardFilterFactory, FieldMutatingUpdateProcessorTest, TestTrie, TestRemoteStreaming]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestRemoteStreaming -Dtests.seed=8E53F6B1A0D0F2CF -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_GT -Dtests.timezone=Pacific/Truk -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] ERROR   0.00s J1 | TestRemoteStreaming (suite) <<<
[junit4:junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming: 
[junit4:junit4]    >    1) Thread[id=297, name=qtp1321261771-297 Acceptor2 SelectChannelConnector@0.0.0.0:58147, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=298, name=qtp1321261771-298 Acceptor3 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=296, name=qtp1321261771-296 Acceptor1 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=295, name=qtp1321261771-295 Acceptor0 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([8E53F6B1A0D0F2CF]:0)
[junit4:junit4]    > Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4]    >    1) Thread[id=297, name=qtp1321261771-297 Acceptor2 SelectChannelConnector@0.0.0.0:58147, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=298, name=qtp1321261771-298 Acceptor3 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=296, name=qtp1321261771-296 Acceptor1 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=295, name=qtp1321261771-295 Acceptor0 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([8E53F6B1A0D0F2CF]:0)
[junit4:junit4] Completed on J1 in 26.42s, 4 tests, 2 errors <<< FAILURES!

[...truncated 122 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:335: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/module-build.xml:61: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:1163: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:827: There were test failures: 233 suites, 965 tests, 4 suite-level errors, 777 ignored

Total time: 18 minutes 35 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure

[JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 607 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.

Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/607/

5 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
12 threads leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:     1) Thread[id=83, name=qtp1635915482-83 Acceptor3 SelectChannelConnector@0.0.0.0:48884, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=80, name=qtp1635915482-80 Acceptor0 SelectChannelConnector@0.0.0.0:48884, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=82, name=qtp1635915482-82 Acceptor2 SelectChannelConnector@0.0.0.0:48884, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=32, name=qtp1013516039-32 Acceptor2 SelectChannelConnector@0.0.0.0:44571, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=33, name=qtp1013516039-33 Acceptor3 SelectChannelConnector@0.0.0.0:44571, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=101, name=qtp370838838-101 Acceptor3 SelectChannelConnector@0.0.0.0:41359, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=100, name=qtp370838838-100 Acceptor2 SelectChannelConnector@0.0.0.0:41359, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=81, name=qtp1635915482-81 Acceptor1 SelectChannelConnector@0.0.0.0:48884, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=99, name=qtp370838838-99 Acceptor1 SelectChannelConnector@0.0.0.0:41359, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=98, name=qtp370838838-98 Acceptor0 SelectChannelConnector@0.0.0.0:41359, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=31, name=qtp1013516039-31 Acceptor1 SelectChannelConnector@0.0.0.0:44571, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=30, name=qtp1013516039-30 Acceptor0 SelectChannelConnector@0.0.0.0:44571, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 12 threads leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest: 
   1) Thread[id=83, name=qtp1635915482-83 Acceptor3 SelectChannelConnector@0.0.0.0:48884, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=80, name=qtp1635915482-80 Acceptor0 SelectChannelConnector@0.0.0.0:48884, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=82, name=qtp1635915482-82 Acceptor2 SelectChannelConnector@0.0.0.0:48884, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=32, name=qtp1013516039-32 Acceptor2 SelectChannelConnector@0.0.0.0:44571, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=33, name=qtp1013516039-33 Acceptor3 SelectChannelConnector@0.0.0.0:44571, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=101, name=qtp370838838-101 Acceptor3 SelectChannelConnector@0.0.0.0:41359, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=100, name=qtp370838838-100 Acceptor2 SelectChannelConnector@0.0.0.0:41359, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=81, name=qtp1635915482-81 Acceptor1 SelectChannelConnector@0.0.0.0:48884, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=99, name=qtp370838838-99 Acceptor1 SelectChannelConnector@0.0.0.0:41359, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=98, name=qtp370838838-98 Acceptor0 SelectChannelConnector@0.0.0.0:41359, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=31, name=qtp1013516039-31 Acceptor1 SelectChannelConnector@0.0.0.0:44571, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=30, name=qtp1013516039-30 Acceptor0 SelectChannelConnector@0.0.0.0:44571, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([97D4AB69C437B5E3]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=100, name=qtp370838838-100 Acceptor2 SelectChannelConnector@0.0.0.0:41359, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=83, name=qtp1635915482-83 Acceptor3 SelectChannelConnector@0.0.0.0:48884, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=80, name=qtp1635915482-80 Acceptor0 SelectChannelConnector@0.0.0.0:48884, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=81, name=qtp1635915482-81 Acceptor1 SelectChannelConnector@0.0.0.0:48884, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=82, name=qtp1635915482-82 Acceptor2 SelectChannelConnector@0.0.0.0:48884, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=99, name=qtp370838838-99 Acceptor1 SelectChannelConnector@0.0.0.0:41359, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=98, name=qtp370838838-98 Acceptor0 SelectChannelConnector@0.0.0.0:41359, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=31, name=qtp1013516039-31 Acceptor1 SelectChannelConnector@0.0.0.0:44571, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=30, name=qtp1013516039-30 Acceptor0 SelectChannelConnector@0.0.0.0:44571, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=32, name=qtp1013516039-32 Acceptor2 SelectChannelConnector@0.0.0.0:44571, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=33, name=qtp1013516039-33 Acceptor3 SelectChannelConnector@0.0.0.0:44571, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=101, name=qtp370838838-101 Acceptor3 SelectChannelConnector@0.0.0.0:41359, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=100, name=qtp370838838-100 Acceptor2 SelectChannelConnector@0.0.0.0:41359, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=83, name=qtp1635915482-83 Acceptor3 SelectChannelConnector@0.0.0.0:48884, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=80, name=qtp1635915482-80 Acceptor0 SelectChannelConnector@0.0.0.0:48884, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=81, name=qtp1635915482-81 Acceptor1 SelectChannelConnector@0.0.0.0:48884, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=82, name=qtp1635915482-82 Acceptor2 SelectChannelConnector@0.0.0.0:48884, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=99, name=qtp370838838-99 Acceptor1 SelectChannelConnector@0.0.0.0:41359, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=98, name=qtp370838838-98 Acceptor0 SelectChannelConnector@0.0.0.0:41359, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=31, name=qtp1013516039-31 Acceptor1 SelectChannelConnector@0.0.0.0:44571, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=30, name=qtp1013516039-30 Acceptor0 SelectChannelConnector@0.0.0.0:44571, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=32, name=qtp1013516039-32 Acceptor2 SelectChannelConnector@0.0.0.0:44571, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=33, name=qtp1013516039-33 Acceptor3 SelectChannelConnector@0.0.0.0:44571, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=101, name=qtp370838838-101 Acceptor3 SelectChannelConnector@0.0.0.0:41359, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([97D4AB69C437B5E3]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.RecoveryZkTest

Error Message:
12 threads leaked from SUITE scope at org.apache.solr.cloud.RecoveryZkTest:     1) Thread[id=217, name=qtp118471033-217 Acceptor3 SelectChannelConnector@0.0.0.0:44503, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=282, name=qtp821896153-282 Acceptor0 SelectChannelConnector@0.0.0.0:48463, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=199, name=qtp574732811-199 Acceptor3 SelectChannelConnector@0.0.0.0:19836, state=RUNNABLE, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=285, name=qtp821896153-285 Acceptor3 SelectChannelConnector@0.0.0.0:48463, state=RUNNABLE, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=198, name=qtp574732811-198 Acceptor2 SelectChannelConnector@0.0.0.0:19836, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=216, name=qtp118471033-216 Acceptor2 SelectChannelConnector@0.0.0.0:44503, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=196, name=qtp574732811-196 Acceptor0 SelectChannelConnector@0.0.0.0:19836, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=215, name=qtp118471033-215 Acceptor1 SelectChannelConnector@0.0.0.0:44503, state=RUNNABLE, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=214, name=qtp118471033-214 Acceptor0 SelectChannelConnector@0.0.0.0:44503, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=283, name=qtp821896153-283 Acceptor1 SelectChannelConnector@0.0.0.0:48463, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=197, name=qtp574732811-197 Acceptor1 SelectChannelConnector@0.0.0.0:19836, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=284, name=qtp821896153-284 Acceptor2 SelectChannelConnector@0.0.0.0:48463, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 12 threads leaked from SUITE scope at org.apache.solr.cloud.RecoveryZkTest: 
   1) Thread[id=217, name=qtp118471033-217 Acceptor3 SelectChannelConnector@0.0.0.0:44503, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=282, name=qtp821896153-282 Acceptor0 SelectChannelConnector@0.0.0.0:48463, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=199, name=qtp574732811-199 Acceptor3 SelectChannelConnector@0.0.0.0:19836, state=RUNNABLE, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=285, name=qtp821896153-285 Acceptor3 SelectChannelConnector@0.0.0.0:48463, state=RUNNABLE, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=198, name=qtp574732811-198 Acceptor2 SelectChannelConnector@0.0.0.0:19836, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=216, name=qtp118471033-216 Acceptor2 SelectChannelConnector@0.0.0.0:44503, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=196, name=qtp574732811-196 Acceptor0 SelectChannelConnector@0.0.0.0:19836, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=215, name=qtp118471033-215 Acceptor1 SelectChannelConnector@0.0.0.0:44503, state=RUNNABLE, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=214, name=qtp118471033-214 Acceptor0 SelectChannelConnector@0.0.0.0:44503, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=283, name=qtp821896153-283 Acceptor1 SelectChannelConnector@0.0.0.0:48463, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=197, name=qtp574732811-197 Acceptor1 SelectChannelConnector@0.0.0.0:19836, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=284, name=qtp821896153-284 Acceptor2 SelectChannelConnector@0.0.0.0:48463, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([97D4AB69C437B5E3]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.RecoveryZkTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=216, name=qtp118471033-216 Acceptor2 SelectChannelConnector@0.0.0.0:44503, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=196, name=qtp574732811-196 Acceptor0 SelectChannelConnector@0.0.0.0:19836, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=215, name=qtp118471033-215 Acceptor1 SelectChannelConnector@0.0.0.0:44503, state=RUNNABLE, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=217, name=qtp118471033-217 Acceptor3 SelectChannelConnector@0.0.0.0:44503, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=282, name=qtp821896153-282 Acceptor0 SelectChannelConnector@0.0.0.0:48463, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=214, name=qtp118471033-214 Acceptor0 SelectChannelConnector@0.0.0.0:44503, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=283, name=qtp821896153-283 Acceptor1 SelectChannelConnector@0.0.0.0:48463, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=199, name=qtp574732811-199 Acceptor3 SelectChannelConnector@0.0.0.0:19836, state=RUNNABLE, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=197, name=qtp574732811-197 Acceptor1 SelectChannelConnector@0.0.0.0:19836, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=284, name=qtp821896153-284 Acceptor2 SelectChannelConnector@0.0.0.0:48463, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=285, name=qtp821896153-285 Acceptor3 SelectChannelConnector@0.0.0.0:48463, state=RUNNABLE, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=198, name=qtp574732811-198 Acceptor2 SelectChannelConnector@0.0.0.0:19836, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=216, name=qtp118471033-216 Acceptor2 SelectChannelConnector@0.0.0.0:44503, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=196, name=qtp574732811-196 Acceptor0 SelectChannelConnector@0.0.0.0:19836, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=215, name=qtp118471033-215 Acceptor1 SelectChannelConnector@0.0.0.0:44503, state=RUNNABLE, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=217, name=qtp118471033-217 Acceptor3 SelectChannelConnector@0.0.0.0:44503, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=282, name=qtp821896153-282 Acceptor0 SelectChannelConnector@0.0.0.0:48463, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=214, name=qtp118471033-214 Acceptor0 SelectChannelConnector@0.0.0.0:44503, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=283, name=qtp821896153-283 Acceptor1 SelectChannelConnector@0.0.0.0:48463, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=199, name=qtp574732811-199 Acceptor3 SelectChannelConnector@0.0.0.0:19836, state=RUNNABLE, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=197, name=qtp574732811-197 Acceptor1 SelectChannelConnector@0.0.0.0:19836, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=284, name=qtp821896153-284 Acceptor2 SelectChannelConnector@0.0.0.0:48463, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=285, name=qtp821896153-285 Acceptor3 SelectChannelConnector@0.0.0.0:48463, state=RUNNABLE, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=198, name=qtp574732811-198 Acceptor2 SelectChannelConnector@0.0.0.0:19836, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([97D4AB69C437B5E3]:0)


FAILED:  org.apache.solr.cloud.RecoveryZkTest.testDistribSearch

Error Message:
Server at http://127.0.0.1:48463/solr/collection1 returned non ok status:404, message:Can not find: /solr/collection1/select

Stack Trace:
org.apache.solr.common.SolrException: Server at http://127.0.0.1:48463/solr/collection1 returned non ok status:404, message:Can not find: /solr/collection1/select
	at __randomizedtesting.SeedInfo.seed([97D4AB69C437B5E3:16322571B368D5DF]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:372)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
	at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:90)
	at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
	at org.apache.solr.cloud.RecoveryZkTest.doTest(RecoveryZkTest.java:92)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:695)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)




Build Log:
[...truncated 8340 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.RecoveryZkTest
[junit4:junit4]   2> 3 T180 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-RecoveryZkTest-1351858862598
[junit4:junit4]   2> 4 T180 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 8 T181 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 8 T181 oazs.NIOServerCnxn$Factory.<init> binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 9 T181 oazsp.FileTxnSnapLog.save Snapshotting: 0
[junit4:junit4]   2> 106 T180 oasc.ZkTestServer.run start zk server on port:20217
[junit4:junit4]   2> 106 T180 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:20217 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@771c674a
[junit4:junit4]   2> 107 T186 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:20217
[junit4:junit4]   2> 107 T180 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 108 T186 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:20217, initiating session
[junit4:junit4]   2> 109 T182 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:59869
[junit4:junit4]   2> 109 T182 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:59869
[junit4:junit4]   2> 110 T184 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 120 T184 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac1113a0c0000 with negotiated timeout 10000 for client /140.211.11.196:59869
[junit4:junit4]   2> 120 T186 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:20217, sessionid = 0x13ac1113a0c0000, negotiated timeout = 10000
[junit4:junit4]   2> 121 T187 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@771c674a name:ZooKeeperConnection Watcher:127.0.0.1:20217 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 121 T180 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 122 T180 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 132 T185 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x13ac1113a0c0000
[junit4:junit4]   2> 142 T187 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 142 T180 oaz.ZooKeeper.close Session: 0x13ac1113a0c0000 closed
[junit4:junit4]   2> 143 T182 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:59869 which had sessionid 0x13ac1113a0c0000
[junit4:junit4]   2> 143 T180 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:20217/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@69dd174a
[junit4:junit4]   2> 144 T188 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:20217
[junit4:junit4]   2> 144 T180 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 145 T188 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:20217, initiating session
[junit4:junit4]   2> 145 T182 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:49570
[junit4:junit4]   2> 145 T182 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:49570
[junit4:junit4]   2> 147 T184 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac1113a0c0001 with negotiated timeout 10000 for client /140.211.11.196:49570
[junit4:junit4]   2> 147 T188 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:20217, sessionid = 0x13ac1113a0c0001, negotiated timeout = 10000
[junit4:junit4]   2> 148 T189 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@69dd174a name:ZooKeeperConnection Watcher:127.0.0.1:20217/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 148 T180 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 148 T180 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 158 T180 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 161 T180 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 163 T180 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 167 T180 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 168 T180 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 198 T180 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 199 T180 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 208 T180 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 209 T180 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 221 T180 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 222 T180 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 228 T180 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 229 T180 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 236 T180 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 237 T180 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 247 T180 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 248 T180 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 257 T180 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 258 T180 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 265 T180 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 265 T180 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 272 T180 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 273 T180 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 276 T185 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x13ac1113a0c0001
[junit4:junit4]   2> 282 T189 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 282 T180 oaz.ZooKeeper.close Session: 0x13ac1113a0c0001 closed
[junit4:junit4]   2> 283 T182 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:49570 which had sessionid 0x13ac1113a0c0001
[junit4:junit4]   2> 864 T180 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 1134 T180 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:19836
[junit4:junit4]   2> 1165 T180 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1166 T180 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 1166 T180 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 1198 T180 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1199 T180 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1199 T180 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 1199 T180 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 1200 T180 oasc.CoreContainer.<init> New CoreContainer 402288927
[junit4:junit4]   2> 1200 T180 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 1201 T180 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 1241 T180 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 1254 T180 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:20217/solr
[junit4:junit4]   2> 1258 T180 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:20217/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@5add9fcc
[junit4:junit4]   2> 1259 T200 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:20217
[junit4:junit4]   2> 1259 T180 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1261 T200 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:20217, initiating session
[junit4:junit4]   2> 1261 T182 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:28076
[junit4:junit4]   2> 1261 T182 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:28076
[junit4:junit4]   2> 1266 T184 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac1113a0c0002 with negotiated timeout 5000 for client /140.211.11.196:28076
[junit4:junit4]   2> 1266 T200 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:20217, sessionid = 0x13ac1113a0c0002, negotiated timeout = 5000
[junit4:junit4]   2> 1267 T201 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5add9fcc name:ZooKeeperConnection Watcher:127.0.0.1:20217/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1267 T180 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1286 T185 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac1113a0c0002 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1304 T180 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 1306 T180 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:19836_solr
[junit4:junit4]   2> 1307 T185 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac1113a0c0002 type:delete cxid:0x7 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:19836_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:19836_solr
[junit4:junit4]   2> 1308 T180 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:19836_solr
[junit4:junit4]   2> 1311 T180 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 1311 T180 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 1311 T180 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 1312 T180 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 1312 T180 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 1312 T180 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 1313 T180 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 1313 T180 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 1313 T180 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 1314 T180 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 1327 T180 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 1335 T185 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac1113a0c0002 type:delete cxid:0x15 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 1336 T180 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 1338 T180 oasc.Overseer.start Overseer (id=88595422419615746-127.0.0.1:19836_solr-n_0000000000) starting
[junit4:junit4]   2> 1339 T185 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac1113a0c0002 type:create cxid:0x1a zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1354 T185 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac1113a0c0002 type:create cxid:0x1b zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1355 T185 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac1113a0c0002 type:create cxid:0x1c zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1360 T185 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac1113a0c0002 type:create cxid:0x1d zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1362 T203 oasc.OverseerCollectionProcessor.run Process current queue of collection messages
[junit4:junit4]   2> 1362 T180 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 1365 T180 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1382 T202 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 1427 T180 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 1427 T180 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 1428 T180 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1429 T180 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 1430 T180 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 1431 T180 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 1432 T180 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 1502 T180 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 1557 T180 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1559 T180 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 1566 T180 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 2013 T180 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 2022 T180 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 2025 T180 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 2044 T180 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2049 T180 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2052 T180 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2054 T180 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2056 T180 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1351858862598/control/data/
[junit4:junit4]   2> 2056 T180 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@13d380dc
[junit4:junit4]   2> 2078 T180 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1351858862598/control/data/index/
[junit4:junit4]   2> 2079 T180 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1351858862598/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2080 T180 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-1351858862598/control/data/index forceNew:false
[junit4:junit4]   2> 2161 T180 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1351858862598/control/data/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 2161 T180 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 2162 T180 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 2164 T180 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2164 T180 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 2165 T180 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2165 T180 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 2166 T180 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2166 T180 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2166 T180 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2167 T180 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2167 T180 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2168 T180 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 2168 T180 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 2169 T180 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2169 T180 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 2170 T180 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 2170 T180 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 2171 T180 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2171 T180 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2172 T180 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2172 T180 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2173 T180 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2173 T180 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2174 T180 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2174 T180 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2175 T180 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 2175 T180 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 2183 T180 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2203 T180 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 2204 T180 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 2208 T180 oass.SolrIndexSearcher.<init> Opening Searcher@396c4f69 main
[junit4:junit4]   2> 2209 T180 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2209 T180 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2209 T180 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 2299 T180 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 2363 T208 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@396c4f69 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2370 T180 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 2372 T185 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac1113a0c0002 type:create cxid:0x53 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 2384 T180 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 2384 T180 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:19836/solr shardId:control_shard
[junit4:junit4]   2> 2388 T202 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2389 T202 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"control_shard",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:19836_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:19836/solr"}
[junit4:junit4]   2> 2404 T180 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 2407 T185 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac1113a0c0002 type:create cxid:0x5b zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 2413 T201 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (1)
[junit4:junit4]   2> 2424 T180 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/control_shard/election
[junit4:junit4]   2> 2431 T185 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac1113a0c0002 type:delete cxid:0x74 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 2432 T180 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 2433 T185 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac1113a0c0002 type:create cxid:0x75 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2435 T180 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 2436 T180 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 2441 T180 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:19836/solr/collection1/
[junit4:junit4]   2> 2442 T180 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 2442 T180 oasc.SyncStrategy.syncToMe http://127.0.0.1:19836/solr/collection1/ has no replicas
[junit4:junit4]   2> 2443 T180 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:19836/solr/collection1/
[junit4:junit4]   2> 2443 T180 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/control_shard
[junit4:junit4]   2> 2509 T185 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac1113a0c0002 type:create cxid:0x7f zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2918 T202 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2940 T201 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (1)
[junit4:junit4]   2> 2984 T180 oasc.ZkController.register We are http://127.0.0.1:19836/solr/collection1/ and leader is http://127.0.0.1:19836/solr/collection1/
[junit4:junit4]   2> 2984 T180 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:19836/solr
[junit4:junit4]   2> 2985 T180 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 2985 T180 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 2988 T180 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2989 T180 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 2989 T180 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3072 T180 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3273 T180 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 3276 T180 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:44503
[junit4:junit4]   2> 3277 T180 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3278 T180 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351858865695
[junit4:junit4]   2> 3278 T180 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351858865695/'
[junit4:junit4]   2> 3331 T180 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 3331 T180 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3332 T180 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351858865695
[junit4:junit4]   2> 3332 T180 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351858865695/solr.xml
[junit4:junit4]   2> 3333 T180 oasc.CoreContainer.<init> New CoreContainer 1145621097
[junit4:junit4]   2> 3333 T180 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351858865695/'
[junit4:junit4]   2> 3334 T180 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351858865695/'
[junit4:junit4]   2> 3369 T180 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 3386 T180 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:20217/solr
[junit4:junit4]   2> 3387 T180 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:20217/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@48ed4623
[junit4:junit4]   2> 3388 T218 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:20217
[junit4:junit4]   2> 3388 T180 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3390 T218 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:20217, initiating session
[junit4:junit4]   2> 3390 T182 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:22292
[junit4:junit4]   2> 3390 T182 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:22292
[junit4:junit4]   2> 3393 T184 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac1113a0c0003 with negotiated timeout 5000 for client /140.211.11.196:22292
[junit4:junit4]   2> 3393 T218 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:20217, sessionid = 0x13ac1113a0c0003, negotiated timeout = 5000
[junit4:junit4]   2> 3393 T219 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@48ed4623 name:ZooKeeperConnection Watcher:127.0.0.1:20217/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3393 T180 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3394 T185 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac1113a0c0003 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3396 T185 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac1113a0c0003 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3407 T180 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44503_solr
[junit4:junit4]   2> 3407 T185 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac1113a0c0003 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:44503_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:44503_solr
[junit4:junit4]   2> 3408 T180 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44503_solr
[junit4:junit4]   2> 3418 T201 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 3418 T180 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 3419 T180 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 3419 T180 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 3420 T180 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 3420 T180 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 3420 T180 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 3421 T180 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 3421 T180 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 3422 T180 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 3422 T180 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 3447 T180 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 3452 T202 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3453 T202 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"control_shard",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:19836_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:19836/solr"}
[junit4:junit4]   2> 3453 T180 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351858865695/collection1
[junit4:junit4]   2> 3454 T180 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 3473 T180 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 3473 T180 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 3476 T180 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351858865695/collection1/'
[junit4:junit4]   2> 3476 T219 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 3476 T201 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 3477 T180 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-jetty1-1351858865695/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 3478 T180 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-jetty1-1351858865695/collection1/lib/README' to classloader
[junit4:junit4]   2> 3538 T180 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 3614 T180 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 3616 T180 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 3626 T180 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 4382 T180 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 4391 T180 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 4394 T180 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 4413 T180 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4418 T180 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4422 T180 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4424 T180 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4425 T180 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351858865695/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1351858862598/jetty1/
[junit4:junit4]   2> 4426 T180 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@13d380dc
[junit4:junit4]   2> 4427 T180 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1351858862598/jetty1/index/
[junit4:junit4]   2> 4428 T180 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1351858862598/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 4428 T180 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-1351858862598/jetty1/index forceNew:false
[junit4:junit4]   2> 4433 T180 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1351858862598/jetty1/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 4433 T180 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 4434 T180 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 4436 T180 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 4436 T180 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 4437 T180 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 4437 T180 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 4438 T180 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 4438 T180 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 4438 T180 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 4439 T180 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 4440 T180 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 4440 T180 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 4441 T180 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 4441 T180 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 4442 T180 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 4442 T180 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 4442 T180 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 4443 T180 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 4444 T180 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4444 T180 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4445 T180 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4445 T180 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4446 T180 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4446 T180 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4447 T180 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4447 T180 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 4448 T180 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 4463 T180 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 4468 T180 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 4469 T180 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 4473 T180 oass.SolrIndexSearcher.<init> Opening Searcher@6f5a6472 main
[junit4:junit4]   2> 4474 T180 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 4474 T180 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 4475 T180 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 4495 T180 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 4567 T224 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6f5a6472 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 4572 T180 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 4993 T202 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4994 T202 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:44503_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:44503/solr"}
[junit4:junit4]   2> 4994 T202 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4]   2> 5022 T201 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 5022 T219 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 5076 T180 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 5076 T180 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:44503/solr shardId:shard1
[junit4:junit4]   2> 5078 T180 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 5106 T185 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac1113a0c0003 type:delete cxid:0x43 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 5108 T180 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 5108 T185 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac1113a0c0003 type:create cxid:0x44 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5119 T180 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 5120 T180 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 5120 T180 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:44503/solr/collection1/
[junit4:junit4]   2> 5120 T180 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 5121 T180 oasc.SyncStrategy.syncToMe http://127.0.0.1:44503/solr/collection1/ has no replicas
[junit4:junit4]   2> 5121 T180 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:44503/solr/collection1/
[junit4:junit4]   2> 5121 T180 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 5134 T185 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac1113a0c0003 type:create cxid:0x4e zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5532 T202 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5556 T201 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 5556 T219 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 5603 T180 oasc.ZkController.register We are http://127.0.0.1:44503/solr/collection1/ and leader is http://127.0.0.1:44503/solr/collection1/
[junit4:junit4]   2> 5603 T180 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44503/solr
[junit4:junit4]   2> 5604 T180 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 5604 T180 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 5607 T180 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5609 T180 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 5609 T180 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 5610 T180 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 5802 T180 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 5805 T180 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:48463
[junit4:junit4]   2> 5806 T180 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 5807 T180 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty2-1351858868222
[junit4:junit4]   2> 5807 T180 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty2-1351858868222/'
[junit4:junit4]   2> 5856 T180 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 5856 T180 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 5857 T180 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty2-1351858868222
[junit4:junit4]   2> 5857 T180 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty2-1351858868222/solr.xml
[junit4:junit4]   2> 5858 T180 oasc.CoreContainer.<init> New CoreContainer 1730644545
[junit4:junit4]   2> 5859 T180 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty2-1351858868222/'
[junit4:junit4]   2> 5859 T180 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty2-1351858868222/'
[junit4:junit4]   2> 5895 T180 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 5912 T180 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:20217/solr
[junit4:junit4]   2> 5913 T180 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:20217/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@a5c6345
[junit4:junit4]   2> 5914 T234 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:20217
[junit4:junit4]   2> 5914 T180 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 5916 T234 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:20217, initiating session
[junit4:junit4]   2> 5916 T182 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:51729
[junit4:junit4]   2> 5916 T182 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:51729
[junit4:junit4]   2> 5918 T184 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac1113a0c0004 with negotiated timeout 5000 for client /140.211.11.196:51729
[junit4:junit4]   2> 5918 T234 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:20217, sessionid = 0x13ac1113a0c0004, negotiated timeout = 5000
[junit4:junit4]   2> 5919 T235 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a5c6345 name:ZooKeeperConnection Watcher:127.0.0.1:20217/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 5919 T180 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 5920 T185 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac1113a0c0004 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5922 T185 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac1113a0c0004 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5924 T180 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48463_solr
[junit4:junit4]   2> 5924 T185 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac1113a0c0004 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:48463_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:48463_solr
[junit4:junit4]   2> 5931 T180 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:48463_solr
[junit4:junit4]   2> 5935 T219 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 5935 T201 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 5936 T180 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 5936 T180 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 5937 T180 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 5937 T180 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 5938 T180 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 5938 T180 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 5939 T180 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 5939 T180 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 5939 T180 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 5940 T180 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 5963 T180 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 5970 T180 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty2-1351858868222/collection1
[junit4:junit4]   2> 5970 T180 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 5971 T180 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 5972 T180 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 5973 T180 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty2-1351858868222/collection1/'
[junit4:junit4]   2> 5975 T180 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-jetty2-1351858868222/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 5975 T180 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-jetty2-1351858868222/collection1/lib/README' to classloader
[junit4:junit4]   2> 6031 T180 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 6069 T202 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6070 T202 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:44503_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:44503/solr"}
[junit4:junit4]   2> 6094 T201 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 6094 T219 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 6094 T235 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 6106 T180 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 6107 T180 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 6118 T180 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 6808 T180 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 6821 T180 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 6826 T180 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 6851 T180 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6858 T180 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6865 T180 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6874 T180 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6876 T180 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty2-1351858868222/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1351858862598/jetty2/
[junit4:junit4]   2> 6877 T180 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@13d380dc
[junit4:junit4]   2> 6879 T180 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1351858862598/jetty2/index/
[junit4:junit4]   2> 6880 T180 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1351858862598/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 6881 T180 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-1351858862598/jetty2/index forceNew:false
[junit4:junit4]   2> 7005 T180 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1351858862598/jetty2/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 7006 T180 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 7006 T180 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 7010 T180 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 7011 T180 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 7011 T180 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 7012 T180 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 7013 T180 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 7013 T180 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 7014 T180 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 7015 T180 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 7016 T180 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 7016 T180 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 7017 T180 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 7018 T180 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 7019 T180 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 7019 T180 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 7020 T180 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 7021 T180 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 7022 T180 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7022 T180 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7023 T180 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7024 T180 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7025 T180 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7026 T180 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7026 T180 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7027 T180 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 7028 T180 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 7041 T180 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 7046 T180 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 7048 T180 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 7052 T180 oass.SolrIndexSearcher.<init> Opening Searcher@2eba74df main
[junit4:junit4]   2> 7053 T180 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 7054 T180 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 7055 T180 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 7163 T180 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 7343 T240 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2eba74df main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 7351 T180 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 7608 T202 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 7609 T202 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:48463_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:48463/solr"}
[junit4:junit4]   2> 7622 T201 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 7622 T235 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 7622 T219 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 7860 T180 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 7860 T180 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:48463/solr shardId:shard1
[junit4:junit4]   2> 7865 T180 oasc.ZkController.register We are http://127.0.0.1:48463/solr/collection1/ and leader is http://127.0.0.1:44503/solr/collection1/
[junit4:junit4]   2> 7865 T180 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:48463/solr
[junit4:junit4]   2> 7866 T180 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 7866 T180 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C20 name=collection1 org.apache.solr.core.SolrCore@28740b0f url=http://127.0.0.1:48463/solr/collection1 node=127.0.0.1:48463_solr
[junit4:junit4]   2> 7874 T241 C20 P48463 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 7874 T180 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 7875 T241 C20 P48463 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 7875 T241 C20 P48463 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 7880 T180 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 7881 T180 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 7881 T180 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 7889 T241 C20 P48463 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:44503/solr/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 7900 T241 C20 P48463 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 7903 T180 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:20217/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@1a301110
[junit4:junit4]   2> 7904 T242 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:20217
[junit4:junit4]   2> 7904 T180 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7905 T242 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:20217, initiating session
[junit4:junit4]   2> 7906 T182 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:64040
[junit4:junit4]   2> 7906 T182 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:64040
[junit4:junit4]   2> 7910 T184 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac1113a0c0005 with negotiated timeout 10000 for client /140.211.11.196:64040
[junit4:junit4]   2> 7910 T242 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:20217, sessionid = 0x13ac1113a0c0005, negotiated timeout = 10000
[junit4:junit4]   2> 7910 T243 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a301110 name:ZooKeeperConnection Watcher:127.0.0.1:20217/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7911 T180 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7912 T180 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 7917 T180 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 7918 T180 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 7922 T241 C20 P48463 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:48463/solr START replicas=[http://127.0.0.1:44503/solr/collection1/] nUpdates=100
[junit4:junit4]   2> 7931 T180 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 7942 T241 C20 P48463 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 7944 T241 C20 P48463 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 7944 T241 C20 P48463 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 7944 T241 C20 P48463 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 8128 T202 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8129 T202 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:48463_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:48463/solr"}
[junit4:junit4]   2> 8140 T243 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 8140 T201 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 8140 T235 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 8140 T219 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> ASYNC  NEW_CORE C21 name=collection1 org.apache.solr.core.SolrCore@1adc811d url=http://127.0.0.1:44503/solr/collection1 node=127.0.0.1:44503_solr
[junit4:junit4]   2> 9224 T220 C21 P44503 REQ /get {distrib=false&qt=/get&wt=javabin&version=2&getVersions=100} status=0 QTime=22 
[junit4:junit4]   2> ASYNC  NEW_CORE C22 name=collection1 org.apache.solr.core.SolrCore@286ee8d5 url=http://127.0.0.1:19836/solr/collection1 node=127.0.0.1:19836_solr
[junit4:junit4]   2> 9288 T207 C22 P19836 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16
[junit4:junit4]   2> 9335 T205 C22 P19836 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1351858862598/control/data/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 9336 T205 C22 P19836 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 9344 T205 C22 P19836 /update {wt=javabin&version=2} {add=[1 (1417526768483434496)]} 0 74
[junit4:junit4]   2> 9344 T207 C22 P19836 /update {wt=javabin&version=2} {add=[10001 (1417526768483434497)]} 0 74
[junit4:junit4]   2> 9395 T245 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:20217/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@58f19e9
[junit4:junit4]   2> 9396 T247 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:20217
[junit4:junit4]   2> 9396 T245 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 9397 T247 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:20217, initiating session
[junit4:junit4]   2> 9397 T182 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:21472
[junit4:junit4]   2> 9398 T182 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:21472
[junit4:junit4]   2> 9400 T184 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac1113a0c0006 with negotiated timeout 10000 for client /140.211.11.196:21472
[junit4:junit4]   2> 9400 T247 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:20217, sessionid = 0x13ac1113a0c0006, negotiated timeout = 10000
[junit4:junit4]   2> 9400 T248 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@58f19e9 name:ZooKeeperConnection Watcher:127.0.0.1:20217/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 9401 T245 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 9401 T245 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 9432 T220 C21 P44503 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1351858862598/jetty1/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 9432 T220 C21 P44503 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2>  C20_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:48463_solr, base_url=http://127.0.0.1:48463/solr}
[junit4:junit4]   2> 9498 T237 C20 P48463 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1351858862598/jetty2/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 9499 T237 C20 P48463 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 9507 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=14 
[junit4:junit4]   2> 9507 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=14 
[junit4:junit4]   2> 9508 T220 C21 P44503 /update {wt=javabin&version=2} {add=[1 (1417526768589340672)]} 0 89
[junit4:junit4]   2> 9508 T223 C21 P44503 /update {wt=javabin&version=2} {add=[10001 (1417526768589340673)]} 0 89
[junit4:junit4]   2> 9516 T207 C22 P19836 /update {wt=javabin&version=2} {add=[10002 (1417526768678469633)]} 0 3
[junit4:junit4]   2> 9516 T206 C22 P19836 /update {wt=javabin&version=2} {add=[2 (1417526768678469632)]} 0 4
[junit4:junit4]   2> 9530 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 9531 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 9531 T222 C21 P44503 /update {wt=javabin&version=2} {add=[2 (1417526768685809664)]} 0 11
[junit4:junit4]   2> 9535 T220 C21 P44503 /update {wt=javabin&version=2} {add=[10002 (1417526768685809665)]} 0 15
[junit4:junit4]   2> 9539 T207 C22 P19836 /update {wt=javabin&version=2} {delete=[10001 (-1417526768705732608)]} 0 0
[junit4:junit4]   2> 9539 T205 C22 P19836 /update {wt=javabin&version=2} {delete=[1 (-1417526768705732609)]} 0 0
[junit4:junit4]   2> 9552 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 9553 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 9554 T222 C21 P44503 /update {wt=javabin&version=2} {delete=[10001 (-1417526768710975488)]} 0 10
[junit4:junit4]   2> 9554 T223 C21 P44503 /update {wt=javabin&version=2} {delete=[1 (-1417526768709926912)]} 0 11
[junit4:junit4]   2> 9561 T207 C22 P19836 /update {wt=javabin&version=2} {add=[10003 (1417526768725655552)]} 0 3
[junit4:junit4]   2> 9561 T206 C22 P19836 /update {wt=javabin&version=2} {add=[3 (1417526768725655553)]} 0 3
[junit4:junit4]   2> 9573 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 9574 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 9575 T220 C21 P44503 /update {wt=javabin&version=2} {add=[3 (1417526768732995584)]} 0 10
[junit4:junit4]   2> 9575 T222 C21 P44503 /update {wt=javabin&version=2} {add=[10003 (1417526768732995585)]} 0 10
[junit4:junit4]   2> 9579 T205 C22 P19836 /update {wt=javabin&version=2} {delete=[2 (-1417526768747675648)]} 0 0
[junit4:junit4]   2> 9579 T207 C22 P19836 /update {wt=javabin&version=2} {delete=[10002 (-1417526768747675649)]} 0 0
[junit4:junit4]   2> 9587 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 9587 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 9588 T223 C21 P44503 /update {wt=javabin&version=2} {delete=[2 (-1417526768751869952)]} 0 5
[junit4:junit4]   2> 9588 T220 C21 P44503 /update {wt=javabin&version=2} {delete=[10002 (-1417526768752918528)]} 0 5
[junit4:junit4]   2> 9594 T205 C22 P19836 /update {wt=javabin&version=2} {add=[4 (1417526768761307137)]} 0 2
[junit4:junit4]   2> 9596 T206 C22 P19836 /update {wt=javabin&version=2} {add=[10004 (1417526768761307136)]} 0 4
[junit4:junit4]   2> 9605 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 9606 T222 C21 P44503 /update {wt=javabin&version=2} {add=[4 (1417526768767598592)]} 0 8
[junit4:junit4]   2> 9610 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 9610 T207 C22 P19836 /update {wt=javabin&version=2} {delete=[3 (-1417526768780181504)]} 0 0
[junit4:junit4]   2> 9611 T223 C21 P44503 /update {wt=javabin&version=2} {add=[10004 (1417526768769695744)]} 0 11
[junit4:junit4]   2> 9615 T205 C22 P19836 /update {wt=javabin&version=2} {delete=[10003 (-1417526768785424384)]} 0 0
[junit4:junit4]   2> 9617 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 9619 T220 C21 P44503 /update {wt=javabin&version=2} {delete=[3 (-1417526768784375808)]} 0 5
[junit4:junit4]   2> 9622 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 9623 T222 C21 P44503 /update {wt=javabin&version=2} {delete=[10003 (-1417526768789618688)]} 0 4
[junit4:junit4]   2> 9626 T206 C22 P19836 /update {wt=javabin&version=2} {add=[5 (1417526768793812992)]} 0 3
[junit4:junit4]   2> 9629 T207 C22 P19836 /update {wt=javabin&version=2} {add=[10005 (1417526768798007296)]} 0 2
[junit4:junit4]   2> 9638 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 9638 T223 C21 P44503 /update {wt=javabin&version=2} {add=[5 (1417526768801153024)]} 0 8
[junit4:junit4]   2> 9642 T205 C22 P19836 /update {wt=javabin&version=2} {delete=[4 (-1417526768813735936)]} 0 0
[junit4:junit4]   2> 9644 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 9645 T220 C21 P44503 /update {wt=javabin&version=2} {add=[10005 (1417526768804298752)]} 0 12
[junit4:junit4]   2> 9649 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 9649 T206 C22 P19836 /update {wt=javabin&version=2} {delete=[10004 (-1417526768821075968)]} 0 0
[junit4:junit4]   2> 9650 T222 C21 P44503 /update {wt=javabin&version=2} {delete=[4 (-1417526768817930240)]} 0 5
[junit4:junit4]   2> 9656 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 9656 T207 C22 P19836 /update {wt=javabin&version=2} {add=[6 (1417526768826318848)]} 0 2
[junit4:junit4]   2> 9657 T223 C21 P44503 /update {wt=javabin&version=2} {delete=[10004 (-1417526768825270272)]} 0 5
[junit4:junit4]   2> 9663 T205 C22 P19836 /update {wt=javabin&version=2} {add=[10006 (1417526768833658880)]} 0 3
[junit4:junit4]   2> 9670 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 9672 T220 C21 P44503 /update {wt=javabin&version=2} {add=[6 (1417526768832610304)]} 0 11
[junit4:junit4]   2> 9677 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 9677 T222 C21 P44503 /update {wt=javabin&version=2} {add=[10006 (1417526768840998912)]} 0 9
[junit4:junit4]   2> 9678 T206 C22 P19836 /update {wt=javabin&version=2} {add=[7 (1417526768849387520)]} 0 2
[junit4:junit4]   2> 9685 T207 C22 P19836 /update {wt=javabin&version=2} {add=[10007 (1417526768855678976)]} 0 3
[junit4:junit4]   2> 9692 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 9693 T223 C21 P44503 /update {wt=javabin&version=2} {add=[7 (1417526768856727552)]} 0 10
[junit4:junit4]   2> 9699 T205 C22 P19836 /update {wt=javabin&version=2} {add=[8 (1417526768871407616)]} 0 2
[junit4:junit4]   2> 9700 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 9701 T220 C21 P44503 /update {wt=javabin&version=2} {add=[10007 (1417526768864067584)]} 0 12
[junit4:junit4]   2> 9711 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 9712 T223 C21 P44503 /update {wt=javabin&version=2} {add=[8 (1417526768877699072)]} 0 9
[junit4:junit4]   2> 9716 T207 C22 P19836 /update {wt=javabin&version=2} {delete=[6 (-1417526768891330560)]} 0 0
[junit4:junit4]   2> 9724 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 9725 T220 C21 P44503 /update {wt=javabin&version=2} {delete=[6 (-1417526768895524864)]} 0 5
[junit4:junit4]   2> 9734 T205 C22 P19836 /update {wt=javabin&version=2} {add=[9 (1417526768904962048)]} 0 5
[junit4:junit4]   2> 9779 T206 C22 P19836 /update {wt=javabin&version=2} {add=[10008 (1417526768880844800)]} 0 73
[junit4:junit4]   2> 9791 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 9792 T220 C21 P44503 /update {wt=javabin&version=2} {add=[10008 (1417526768961585152)]} 0 9
[junit4:junit4]   2> 9796 T207 C22 P19836 /update {wt=javabin&version=2} {delete=[10006 (-1417526768975216640)]} 0 1
[junit4:junit4]   2> 9804 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 9805 T220 C21 P44503 /update {wt=javabin&version=2} {delete=[10006 (-1417526768979410944)]} 0 5
[junit4:junit4]   2> 9811 T205 C22 P19836 /update {wt=javabin&version=2} {add=[10009 (1417526768988848128)]} 0 2
[junit4:junit4]   2> 9825 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 9826 T223 C21 P44503 /update {wt=javabin&version=2} {add=[10009 (1417526768995139584)]} 0 11
[junit4:junit4]   2> 9834 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=58 
[junit4:junit4]   2> 9834 T206 C22 P19836 /update {wt=javabin&version=2} {add=[10010 (1417526769011916800)]} 0 3
[junit4:junit4]   2> 9835 T222 C21 P44503 /update {wt=javabin&version=2} {add=[9 (1417526768915447808)]} 0 96
[junit4:junit4]   2> 9841 T207 C22 P19836 /update {wt=javabin&version=2} {add=[10 (1417526769020305408)]} 0 2
[junit4:junit4]   2> 9847 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 9848 T220 C21 P44503 /update {wt=javabin&version=2} {add=[10010 (1417526769020305408)]} 0 9
[junit4:junit4]   2> 9854 T205 C22 P19836 /update {wt=javabin&version=2} {add=[10011 (1417526769033936896)]} 0 2
[junit4:junit4]   2> 9856 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=6 
[junit4:junit4]   2> 9858 T223 C21 P44503 /update {wt=javabin&version=2} {add=[10 (1417526769026596864)]} 0 13
[junit4:junit4]   2> 9864 T206 C22 P19836 /update {wt=javabin&version=2} {add=[11 (1417526769044422656)]} 0 2
[junit4:junit4]   2> 9876 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 9877 T220 C21 P44503 /update {wt=javabin&version=2} {add=[11 (1417526769050714112)]} 0 9
[junit4:junit4]   2> 9881 T207 C22 P19836 /update {wt=javabin&version=2} {delete=[7 (-1417526769064345600)]} 0 0
[junit4:junit4]   2> 9888 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 9890 T223 C21 P44503 /update {wt=javabin&version=2} {delete=[7 (-1417526769068539904)]} 0 5
[junit4:junit4]   2> 9897 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=32 
[junit4:junit4]   2> 9897 T205 C22 P19836 /update {wt=javabin&version=2} {add=[12 (1417526769077977088)]} 0 3
[junit4:junit4]   2> 9898 T222 C21 P44503 /update {wt=javabin&version=2} {add=[10011 (1417526769041276928)]} 0 39
[junit4:junit4]   2> 9903 T206 C22 P19836 /update {wt=javabin&version=2} {delete=[10007 (-1417526769087414272)]} 0 0
[junit4:junit4]   2> 9911 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 9912 T223 C21 P44503 /update {wt=javabin&version=2} {delete=[10007 (-1417526769092657152)]} 0 4
[junit4:junit4]   2> 9918 T207 C22 P19836 /update {wt=javabin&version=2} {add=[10012 (1417526769101045760)]} 0 2
[junit4:junit4]   2> 9933 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 9934 T222 C21 P44503 /update {wt=javabin&version=2} {add=[10012 (1417526769108385792)]} 0 11
[junit4:junit4]   2> 9937 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 9938 T220 C21 P44503 /update {wt=javabin&version=2} {add=[12 (1417526769086365696)]} 0 36
[junit4:junit4]   2> 9940 T206 C22 P19836 /update {wt=javabin&version=2} {add=[10013 (1417526769124114432)]} 0 3
[junit4:junit4]   2> 9944 T205 C22 P19836 /update {wt=javabin&version=2} {add=[13 (1417526769128308736)]} 0 2
[junit4:junit4]   2> 9951 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 9952 T223 C21 P44503 /update {wt=javabin&version=2} {add=[10013 (1417526769130405888)]} 0 9
[junit4:junit4]   2> 9955 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 9956 T222 C21 P44503 /update {wt=javabin&version=2} {add=[13 (1417526769134600192)]} 0 9
[junit4:junit4]   2> 9958 T205 C22 P19836 /update {wt=javabin&version=2} {add=[10014 (1417526769142988800)]} 0 3
[junit4:junit4]   2> 9962 T207 C22 P19836 /update {wt=javabin&version=2} {add=[14 (1417526769147183104)]} 0 2
[junit4:junit4]   2> 9973 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 9974 T220 C21 P44503 /update {wt=javabin&version=2} {add=[10014 (1417526769149280256)]} 0 13
[junit4:junit4]   2> 9975 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 9976 T223 C21 P44503 /update {wt=javabin&version=2} {add=[14 (1417526769154523136)]} 0 9
[junit4:junit4]   2> 9986 T205 C22 P19836 /update {wt=javabin&version=2} {add=[15 (1417526769169203200)]} 0 6
[junit4:junit4]   2> 9998 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 9999 T222 C21 P44503 /update {wt=javabin&version=2} {add=[15 (1417526769178640384)]} 0 9
[junit4:junit4]   2> 10004 T207 C22 P19836 /update {wt=javabin&version=2} {delete=[10 (-1417526769193320448)]} 0 0
[junit4:junit4]   2> 10011 T206 C22 P19836 /update {wt=javabin&version=2} {add=[10015 (1417526769167106048)]} 0 32
[junit4:junit4]   2> 10011 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 10012 T220 C21 P44503 /update {wt=javabin&version=2} {delete=[10 (-1417526769197514752)]} 0 4
[junit4:junit4]   2> 10019 T205 C22 P19836 /update {wt=javabin&version=2} {add=[16 (1417526769205903360)]} 0 3
[junit4:junit4]   2> 10023 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10023 T223 C21 P44503 /update {wt=javabin&version=2} {add=[10015 (1417526769204854784)]} 0 9
[junit4:junit4]   2> 10027 T207 C22 P19836 /update {wt=javabin&version=2} {delete=[10010 (-1417526769217437696)]} 0 0
[junit4:junit4]   2> 10035 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 10036 T220 C21 P44503 /update {wt=javabin&version=2} {delete=[10010 (-1417526769221632000)]} 0 5
[junit4:junit4]   2> 10044 T206 C22 P19836 /update {wt=javabin&version=2} {add=[10016 (1417526769232117760)]} 0 3
[junit4:junit4]   2> 10055 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 10056 T222 C21 P44503 /update {wt=javabin&version=2} {add=[16 (1417526769213243392)]} 0 33
[junit4:junit4]   2> 10059 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10059 T223 C21 P44503 /update {wt=javabin&version=2} {add=[10016 (1417526769239457792)]} 0 11
[junit4:junit4]   2> 10061 T205 C22 P19836 /update {wt=javabin&version=2} {delete=[11 (-1417526769253089280)]} 0 0
[junit4:junit4]   2> 10063 T207 C22 P19836 /update {wt=javabin&version=2} {delete=[10011 (-1417526769255186432)]} 0 1
[junit4:junit4]   2> 10069 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 10070 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 10070 T220 C21 P44503 /update {wt=javabin&version=2} {delete=[11 (-1417526769257283584)]} 0 5
[junit4:junit4]   2> 10071 T222 C21 P44503 /update {wt=javabin&version=2} {delete=[10011 (-1417526769259380736)]} 0 4
[junit4:junit4]   2> 10076 T206 C22 P19836 /update {wt=javabin&version=2} {add=[17 (1417526769266720768)]} 0 3
[junit4:junit4]   2> 10078 T205 C22 P19836 /update {wt=javabin&version=2} {add=[10017 (1417526769267769344)]} 0 3
[junit4:junit4]   2> 10091 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 10092 T220 C21 P44503 /update {wt=javabin&version=2} {add=[10017 (1417526769275109376)]} 0 10
[junit4:junit4]   2> 10096 T207 C22 P19836 /update {wt=javabin&version=2} {delete=[10015 (-1417526769289789440)]} 0 0
[junit4:junit4]   2> 10103 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 10104 T222 C21 P44503 /update {wt=javabin&version=2} {delete=[10015 (-1417526769293983744)]} 0 5
[junit4:junit4]   2> 10110 T206 C22 P19836 /update {wt=javabin&version=2} {add=[10018 (1417526769302372352)]} 0 2
[junit4:junit4]   2> 10113 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=28 
[junit4:junit4]   2> 10114 T223 C21 P44503 /update {wt=javabin&version=2} {add=[17 (1417526769273012224)]} 0 35
[junit4:junit4]   2> 10118 T205 C22 P19836 /update {wt=javabin&version=2} {delete=[15 (-1417526769312858112)]} 0 1
[junit4:junit4]   2> 10121 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 10122 T222 C21 P44503 /update {wt=javabin&version=2} {add=[10018 (1417526769308663808)]} 0 9
[junit4:junit4]   2> 10125 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 10126 T223 C21 P44503 /update {wt=javabin&version=2} {delete=[15 (-1417526769317052416)]} 0 5
[junit4:junit4]   2> 10132 T206 C22 P19836 /update {wt=javabin&version=2} {add=[18 (1417526769325441024)]} 0 2
[junit4:junit4]   2> 10146 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10147 T220 C21 P44503 /update {wt=javabin&version=2} {add=[18 (1417526769332781056)]} 0 11
[junit4:junit4]   2> 10153 T205 C22 P19836 /update {wt=javabin&version=2} {add=[19 (1417526769347461120)]} 0 2
[junit4:junit4]   2> 10164 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 10165 T222 C21 P44503 /update {wt=javabin&version=2} {add=[19 (1417526769353752576)]} 0 9
[junit4:junit4]   2> 10166 T207 C22 P19836 /update {wt=javabin&version=2} {add=[10019 (1417526769321246720)]} 0 40
[junit4:junit4]   2> 10170 T206 C22 P19836 /update {wt=javabin&version=2} {delete=[16 (-1417526769367384064)]} 0 0
[junit4:junit4]   2> 10178 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 10179 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 10179 T223 C21 P44503 /update {wt=javabin&version=2} {delete=[16 (-1417526769372626944)]} 0 4
[junit4:junit4]   2> 10180 T220 C21 P44503 /update {wt=javabin&version=2} {add=[10019 (1417526769368432640)]} 0 10
[junit4:junit4]   2> 10184 T205 C22 P19836 /update {wt=javabin&version=2} {delete=[10016 (-1417526769382064128)]} 0 0
[junit4:junit4]   2> 10190 T207 C22 P19836 /update {wt=javabin&version=2} {add=[20 (1417526769382064129)]} 0 6
[junit4:junit4]   2> 10191 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 10192 T222 C21 P44503 /update {wt=javabin&version=2} {delete=[10016 (-1417526769386258432)]} 0 4
[junit4:junit4]   2> 10198 T206 C22 P19836 /update {wt=javabin&version=2} {add=[10020 (1417526769394647040)]} 0 3
[junit4:junit4]   2> 10201 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 10203 T223 C21 P44503 /update {wt=javabin&version=2} {add=[20 (1417526769392549888)]} 0 9
[junit4:junit4]   2> 10210 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10211 T222 C21 P44503 /update {wt=javabin&version=2} {add=[10020 (1417526769400938496)]} 0 9
[junit4:junit4]   2> 10217 T207 C22 P19836 /update {wt=javabin&version=2} {add=[10021 (1417526769414569984)]} 0 3
[junit4:junit4]   2> 10229 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 10230 T220 C21 P44503 /update {wt=javabin&version=2} {add=[10021 (1417526769421910016)]} 0 9
[junit4:junit4]   2> 10234 T205 C22 P19836 /update {wt=javabin&version=2} {add=[21 (1417526769406181376)]} 0 28
[junit4:junit4]   2> 10234 T206 C22 P19836 /update {wt=javabin&version=2} {delete=[10021 (-1417526769434492928)]} 0 1
[junit4:junit4]   2> 10241 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 10242 T222 C21 P44503 /update {wt=javabin&version=2} {delete=[10021 (-1417526769438687232)]} 0 5
[junit4:junit4]   2> 10245 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 10246 T223 C21 P44503 /update {wt=javabin&version=2} {add=[21 (1417526769438687233)]} 0 9
[junit4:junit4]   2> 10248 T207 C22 P19836 /update {wt=javabin&version=2} {add=[10022 (1417526769447075840)]} 0 3
[junit4:junit4]   2> 10250 T205 C22 P19836 /update {wt=javabin&version=2} {delete=[21 (-1417526769451270144)]} 0 1
[junit4:junit4]   2> 10257 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 10258 T222 C21 P44503 /update {wt=javabin&version=2} {delete=[21 (-1417526769455464448)]} 0 5
[junit4:junit4]   2> 10259 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10259 T220 C21 P44503 /update {wt=javabin&version=2} {add=[10022 (1417526769453367296)]} 0 8
[junit4:junit4]   2> 10263 T207 C22 P19836 /update {wt=javabin&version=2} {delete=[10022 (-1417526769464901632)]} 0 1
[junit4:junit4]   2> 10265 T206 C22 P19836 /update {wt=javabin&version=2} {add=[22 (1417526769463853056)]} 0 4
[junit4:junit4]   2> 10270 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 10271 T223 C21 P44503 /update {wt=javabin&version=2} {delete=[10022 (-1417526769469095936)]} 0 5
[junit4:junit4]   2> 10276 T205 C22 P19836 /update {wt=javabin&version=2} {add=[10023 (1417526769476435968)]} 0 3
[junit4:junit4]   2> 10276 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 10277 T222 C21 P44503 /update {wt=javabin&version=2} {add=[22 (1417526769471193088)]} 0 9
[junit4:junit4]   2> 10281 T207 C22 P19836 /update {wt=javabin&version=2} {delete=[22 (-1417526769482727424)]} 0 1
[junit4:junit4]   2> 10287 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10288 T220 C21 P44503 /update {wt=javabin&version=2} {add=[10023 (1417526769482727424)]} 0 9
[junit4:junit4]   2> 10288 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 10289 T223 C21 P44503 /update {wt=javabin&version=2} {delete=[22 (-1417526769487970304)]} 0 5
[junit4:junit4]   2> 10292 T206 C22 P19836 /update {wt=javabin&version=2} {add=[10024 (1417526769494261760)]} 0 2
[junit4:junit4]   2> 10294 T205 C22 P19836 /update {wt=javabin&version=2} {add=[23 (1417526769496358912)]} 0 2
[junit4:junit4]   2> 10304 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10304 T222 C21 P44503 /update {wt=javabin&version=2} {add=[10024 (1417526769499504640)]} 0 8
[junit4:junit4]   2> 10309 T207 C22 P19836 /update {wt=javabin&version=2} {delete=[10023 (-1417526769513136128)]} 0 0
[junit4:junit4]   2> 10316 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 10317 T223 C21 P44503 /update {wt=javabin&version=2} {delete=[10023 (-1417526769517330432)]} 0 5
[junit4:junit4]   2> 10323 T206 C22 P19836 /update {wt=javabin&version=2} {add=[10025 (1417526769525719040)]} 0 3
[junit4:junit4]   2> 10328 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=26 
[junit4:junit4]   2> 10329 T220 C21 P44503 /update {wt=javabin&version=2} {add=[23 (1417526769501601792)]} 0 32
[junit4:junit4]   2> 10335 T207 C22 P19836 /update {wt=javabin&version=2} {add=[24 (1417526769538301952)]} 0 3
[junit4:junit4]   2> 10348 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 10349 T223 C21 P44503 /update {wt=javabin&version=2} {add=[24 (1417526769544593408)]} 0 11
[junit4:junit4]   2> 10353 T205 C22 P19836 /update {wt=javabin&version=2} {delete=[23 (-1417526769559273472)]} 0 0
[junit4:junit4]   2> 10360 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 10361 T220 C21 P44503 /update {wt=javabin&version=2} {delete=[23 (-1417526769563467776)]} 0 4
[junit4:junit4]   2> 10364 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 10365 T222 C21 P44503 /update {wt=javabin&version=2} {add=[10025 (1417526769532010496)]} 0 39
[junit4:junit4]   2> 10367 T206 C22 P19836 /update {wt=javabin&version=2} {add=[25 (1417526769571856384)]} 0 3
[junit4:junit4]   2> 10369 T207 C22 P19836 /update {wt=javabin&version=2} {delete=[10024 (-1417526769576050688)]} 0 1
[junit4:junit4]   2> 10376 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 10377 T220 C21 P44503 /update {wt=javabin&version=2} {delete=[10024 (-1417526769580244992)]} 0 5
[junit4:junit4]   2> 10377 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10378 T223 C21 P44503 /update {wt=javabin&version=2} {add=[25 (1417526769577099264)]} 0 9
[junit4:junit4]   2> 10382 T205 C22 P19836 /update {wt=javabin&version=2} {delete=[24 (-1417526769589682176)]} 0 1
[junit4:junit4]   2> 10383 T206 C22 P19836 /update {wt=javabin&version=2} {add=[10026 (1417526769588633600)]} 0 2
[junit4:junit4]   2> 10389 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 10390 T220 C21 P44503 /update {wt=javabin&version=2} {delete=[24 (-1417526769593876480)]} 0 4
[junit4:junit4]   2> 10393 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10394 T222 C21 P44503 /update {wt=javabin&version=2} {add=[10026 (1417526769593876481)]} 0 8
[junit4:junit4]   2> 10395 T207 C22 P19836 /update {wt=javabin&version=2} {add=[26 (1417526769602265088)]} 0 2
[junit4:junit4]   2> 10399 T205 C22 P19836 /update {wt=javabin&version=2} {add=[10027 (1417526769606459392)]} 0 2
[junit4:junit4]   2> 10410 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10410 T220 C21 P44503 /update {wt=javabin&version=2} {add=[10027 (1417526769611702272)]} 0 7
[junit4:junit4]   2> 10417 T206 C22 P19836 /update {wt=javabin&version=2} {add=[10028 (1417526769624285184)]} 0 3
[junit4:junit4]   2> 10427 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 10428 T222 C21 P44503 /update {wt=javabin&version=2} {add=[10028 (1417526769630576640)]} 0 8
[junit4:junit4]   2> 10434 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=30 
[junit4:junit4]   2> 10434 T207 C22 P19836 /update {wt=javabin&version=2} {add=[10029 (1417526769642110976)]} 0 3
[junit4:junit4]   2> 10435 T223 C21 P44503 /update {wt=javabin&version=2} {add=[26 (1417526769608556544)]} 0 35
[junit4:junit4]   2> 10470 T205 C22 P19836 /update {wt=javabin&version=2} {add=[27 (1417526769648402432)]} 0 33
[junit4:junit4]   2> 10477 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 10478 T220 C21 P44503 /update {wt=javabin&version=2} {add=[10029 (1417526769648402432)]} 0 41
[junit4:junit4]   2> 10482 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10483 T222 C21 P44503 /update {wt=javabin&version=2} {add=[27 (1417526769687199744)]} 0 8
[junit4:junit4]   2> 10485 T206 C22 P19836 /update {wt=javabin&version=2} {add=[10030 (1417526769694539776)]} 0 4
[junit4:junit4]   2> 10490 T207 C22 P19836 /update {wt=javabin&version=2} {add=[28 (1417526769699782656)]} 0 4
[junit4:junit4]   2> 10498 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 10499 T223 C21 P44503 /update {wt=javabin&version=2} {add=[10030 (1417526769701879808)]} 0 11
[junit4:junit4]   2> 10503 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10503 T220 C21 P44503 /update {wt=javabin&version=2} {add=[28 (1417526769707122688)]} 0 10
[junit4:junit4]   2> 10505 T205 C22 P19836 /update {wt=javabin&version=2} {add=[10031 (1417526769716559872)]} 0 3
[junit4:junit4]   2> 10508 T206 C22 P19836 /update {wt=javabin&version=2} {add=[29 (1417526769720754176)]} 0 2
[junit4:junit4]   2> 10517 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 10518 T222 C21 P44503 /update {wt=javabin&version=2} {add=[10031 (1417526769722851328)]} 0 9
[junit4:junit4]   2> 10519 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10519 T220 C21 P44503 /update {wt=javabin&version=2} {add=[29 (1417526769725997056)]} 0 8
[junit4:junit4]   2> 10524 T207 C22 P19836 /update {wt=javabin&version=2} {add=[10032 (1417526769736482816)]} 0 3
[junit4:junit4]   2> 10524 T205 C22 P19836 /update {wt=javabin&version=2} {add=[30 (1417526769737531392)]} 0 2
[junit4:junit4]   2> 10535 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 10536 T220 C21 P44503 /update {wt=javabin&version=2} {add=[30 (1417526769742774273)]} 0 9
[junit4:junit4]   2> 10542 T206 C22 P19836 /update {wt=javabin&version=2} {add=[31 (1417526769755357184)]} 0 3
[junit4:junit4]   2> 10561 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=28 
[junit4:junit4]   2> 10563 T222 C21 P44503 /update {wt=javabin&version=2} {add=[10032 (1417526769742774272)]} 0 36
[junit4:junit4]   2> 10569 T207 C22 P19836 /update {wt=javabin&version=2} {add=[10033 (1417526769783668736)]} 0 2
[junit4:junit4]   2> 10577 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 10578 T223 C21 P44503 /update {wt=javabin&version=2} {add=[31 (1417526769761648640)]} 0 33
[junit4:junit4]   2> 10582 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 10583 T220 C21 P44503 /update {wt=javabin&version=2} {add=[10033 (1417526769789960192)]} 0 11
[junit4:junit4]   2> 10584 T205 C22 P19836 /update {wt=javabin&version=2} {add=[32 (1417526769799397376)]} 0 3
[junit4:junit4]   2> 10587 T206 C22 P19836 /update {wt=javabin&version=2} {delete=[10028 (-1417526769804640256)]} 0 1
[junit4:junit4]   2> 10594 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 10595 T223 C21 P44503 /update {wt=javabin&version=2} {delete=[10028 (-1417526769808834560)]} 0 5
[junit4:junit4]   2> 10597 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10598 T222 C21 P44503 /update {wt=javabin&version=2} {add=[32 (1417526769805688832)]} 0 11
[junit4:junit4]   2> 10604 T205 C22 P19836 /update {wt=javabin&version=2} {add=[33 (1417526769820368896)]} 0 3
[junit4:junit4]   2> 10617 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10618 T223 C21 P44503 /update {wt=javabin&version=2} {add=[33 (1417526769827708928)]} 0 9
[junit4:junit4]   2> 10622 T206 C22 P19836 /update {wt=javabin&version=2} {delete=[28 (-1417526769841340416)]} 0 0
[junit4:junit4]   2> 10628 T207 C22 P19836 /update {wt=javabin&version=2} {add=[10034 (1417526769817223168)]} 0 29
[junit4:junit4]   2> 10629 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 10630 T220 C21 P44503 /update {wt=javabin&version=2} {delete=[28 (-1417526769845534720)]} 0 5
[junit4:junit4]   2> 10636 T205 C22 P19836 /update {wt=javabin&version=2} {add=[34 (1417526769853923328)]} 0 3
[junit4:junit4]   2> 10639 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10640 T222 C21 P44503 /update {wt=javabin&version=2} {add=[10034 (1417526769851826176)]} 0 9
[junit4:junit4]   2> 10644 T206 C22 P19836 /update {wt=javabin&version=2} {delete=[10029 (-1417526769864409088)]} 0 1
[junit4:junit4]   2> 10647 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10648 T223 C21 P44503 /update {wt=javabin&version=2} {add=[34 (1417526769860214784)]} 0 8
[junit4:junit4]   2> 10652 T207 C22 P19836 /update {wt=javabin&version=2} {delete=[29 (-1417526769872797696)]} 0 1
[junit4:junit4]   2> 10654 T237 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 10655 T220 C21 P44503 /update {wt=javabin&version=2} {delete=[10029 (-1417526769870700544)]} 0 5
[junit4:junit4]   2> 10659 T239 C20 P48463 REQ /update {distrib.from=http://127.0.0.1:44503/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 10660 T222 C21 P44503 /update {wt=javabin&version=2} {delete=[29 (-1417526769876992000)]} 0 5
[junit4:junit4]   2> 10661 T205 C22 P19836 /update {wt=javabin&version=2} {add=[10035 (1417526769880137728)]} 0 2
[junit4:junit4]   2> 10673 T238 C20 P48463 REQ /update {distrib.from=http://127.0.0

[...truncated too long message...]

ctChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=80, name=qtp1635915482-80 Acceptor0 SelectChannelConnector@0.0.0.0:48884, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=81, name=qtp1635915482-81 Acceptor1 SelectChannelConnector@0.0.0.0:48884, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    5) Thread[id=82, name=qtp1635915482-82 Acceptor2 SelectChannelConnector@0.0.0.0:48884, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    6) Thread[id=99, name=qtp370838838-99 Acceptor1 SelectChannelConnector@0.0.0.0:41359, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    7) Thread[id=98, name=qtp370838838-98 Acceptor0 SelectChannelConnector@0.0.0.0:41359, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    8) Thread[id=31, name=qtp1013516039-31 Acceptor1 SelectChannelConnector@0.0.0.0:44571, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    9) Thread[id=30, name=qtp1013516039-30 Acceptor0 SelectChannelConnector@0.0.0.0:44571, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   10) Thread[id=32, name=qtp1013516039-32 Acceptor2 SelectChannelConnector@0.0.0.0:44571, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   11) Thread[id=33, name=qtp1013516039-33 Acceptor3 SelectChannelConnector@0.0.0.0:44571, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   12) Thread[id=101, name=qtp370838838-101 Acceptor3 SelectChannelConnector@0.0.0.0:41359, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([97D4AB69C437B5E3]:0)
[junit4:junit4] Completed on J0 in 291.79s, 1 test, 2 errors <<< FAILURES!

[...truncated 180 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:335: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/module-build.xml:61: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:1163: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:827: There were test failures: 233 suites, 965 tests, 4 suite-level errors, 1 error, 825 ignored

Total time: 21 minutes 35 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure

[JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 606 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.

Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/606/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.handler.component.DistributedQueryElevationComponentTest

Error Message:
16 threads leaked from SUITE scope at org.apache.solr.handler.component.DistributedQueryElevationComponentTest:     1) Thread[id=392, name=qtp151042765-392 Acceptor3 SelectChannelConnector@0.0.0.0:46316, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=390, name=qtp151042765-390 Acceptor1 SelectChannelConnector@0.0.0.0:46316, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=352, name=qtp1913461020-352 Acceptor1 SelectChannelConnector@0.0.0.0:46304, state=RUNNABLE, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=365, name=qtp1557962933-365 Acceptor0 SelectChannelConnector@0.0.0.0:46311, state=RUNNABLE, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=389, name=qtp151042765-389 Acceptor0 SelectChannelConnector@0.0.0.0:46316, state=RUNNABLE, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=367, name=qtp1557962933-367 Acceptor2 SelectChannelConnector@0.0.0.0:46311, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=353, name=qtp1913461020-353 Acceptor2 SelectChannelConnector@0.0.0.0:46304, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=381, name=qtp1963244738-381 Acceptor2 SelectChannelConnector@0.0.0.0:46312, state=RUNNABLE, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=351, name=qtp1913461020-351 Acceptor0 SelectChannelConnector@0.0.0.0:46304, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=380, name=qtp1963244738-380 Acceptor1 SelectChannelConnector@0.0.0.0:46312, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=391, name=qtp151042765-391 Acceptor2 SelectChannelConnector@0.0.0.0:46316, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=368, name=qtp1557962933-368 Acceptor3 SelectChannelConnector@0.0.0.0:46311, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=354, name=qtp1913461020-354 Acceptor3 SelectChannelConnector@0.0.0.0:46304, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=366, name=qtp1557962933-366 Acceptor1 SelectChannelConnector@0.0.0.0:46311, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=379, name=qtp1963244738-379 Acceptor0 SelectChannelConnector@0.0.0.0:46312, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=382, name=qtp1963244738-382 Acceptor3 SelectChannelConnector@0.0.0.0:46312, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 16 threads leaked from SUITE scope at org.apache.solr.handler.component.DistributedQueryElevationComponentTest: 
   1) Thread[id=392, name=qtp151042765-392 Acceptor3 SelectChannelConnector@0.0.0.0:46316, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=390, name=qtp151042765-390 Acceptor1 SelectChannelConnector@0.0.0.0:46316, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=352, name=qtp1913461020-352 Acceptor1 SelectChannelConnector@0.0.0.0:46304, state=RUNNABLE, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=365, name=qtp1557962933-365 Acceptor0 SelectChannelConnector@0.0.0.0:46311, state=RUNNABLE, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=389, name=qtp151042765-389 Acceptor0 SelectChannelConnector@0.0.0.0:46316, state=RUNNABLE, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=367, name=qtp1557962933-367 Acceptor2 SelectChannelConnector@0.0.0.0:46311, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=353, name=qtp1913461020-353 Acceptor2 SelectChannelConnector@0.0.0.0:46304, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=381, name=qtp1963244738-381 Acceptor2 SelectChannelConnector@0.0.0.0:46312, state=RUNNABLE, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=351, name=qtp1913461020-351 Acceptor0 SelectChannelConnector@0.0.0.0:46304, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=380, name=qtp1963244738-380 Acceptor1 SelectChannelConnector@0.0.0.0:46312, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=391, name=qtp151042765-391 Acceptor2 SelectChannelConnector@0.0.0.0:46316, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=368, name=qtp1557962933-368 Acceptor3 SelectChannelConnector@0.0.0.0:46311, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=354, name=qtp1913461020-354 Acceptor3 SelectChannelConnector@0.0.0.0:46304, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=366, name=qtp1557962933-366 Acceptor1 SelectChannelConnector@0.0.0.0:46311, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=379, name=qtp1963244738-379 Acceptor0 SelectChannelConnector@0.0.0.0:46312, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=382, name=qtp1963244738-382 Acceptor3 SelectChannelConnector@0.0.0.0:46312, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([398794AF1A5D8DB8]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.component.DistributedQueryElevationComponentTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=392, name=qtp151042765-392 Acceptor3 SelectChannelConnector@0.0.0.0:46316, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=352, name=qtp1913461020-352 Acceptor1 SelectChannelConnector@0.0.0.0:46304, state=RUNNABLE, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=390, name=qtp151042765-390 Acceptor1 SelectChannelConnector@0.0.0.0:46316, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=365, name=qtp1557962933-365 Acceptor0 SelectChannelConnector@0.0.0.0:46311, state=RUNNABLE, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=389, name=qtp151042765-389 Acceptor0 SelectChannelConnector@0.0.0.0:46316, state=RUNNABLE, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=353, name=qtp1913461020-353 Acceptor2 SelectChannelConnector@0.0.0.0:46304, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=367, name=qtp1557962933-367 Acceptor2 SelectChannelConnector@0.0.0.0:46311, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=381, name=qtp1963244738-381 Acceptor2 SelectChannelConnector@0.0.0.0:46312, state=RUNNABLE, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=351, name=qtp1913461020-351 Acceptor0 SelectChannelConnector@0.0.0.0:46304, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=380, name=qtp1963244738-380 Acceptor1 SelectChannelConnector@0.0.0.0:46312, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=368, name=qtp1557962933-368 Acceptor3 SelectChannelConnector@0.0.0.0:46311, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=391, name=qtp151042765-391 Acceptor2 SelectChannelConnector@0.0.0.0:46316, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=354, name=qtp1913461020-354 Acceptor3 SelectChannelConnector@0.0.0.0:46304, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=366, name=qtp1557962933-366 Acceptor1 SelectChannelConnector@0.0.0.0:46311, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=379, name=qtp1963244738-379 Acceptor0 SelectChannelConnector@0.0.0.0:46312, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=382, name=qtp1963244738-382 Acceptor3 SelectChannelConnector@0.0.0.0:46312, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=392, name=qtp151042765-392 Acceptor3 SelectChannelConnector@0.0.0.0:46316, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=352, name=qtp1913461020-352 Acceptor1 SelectChannelConnector@0.0.0.0:46304, state=RUNNABLE, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=390, name=qtp151042765-390 Acceptor1 SelectChannelConnector@0.0.0.0:46316, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=365, name=qtp1557962933-365 Acceptor0 SelectChannelConnector@0.0.0.0:46311, state=RUNNABLE, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=389, name=qtp151042765-389 Acceptor0 SelectChannelConnector@0.0.0.0:46316, state=RUNNABLE, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=353, name=qtp1913461020-353 Acceptor2 SelectChannelConnector@0.0.0.0:46304, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=367, name=qtp1557962933-367 Acceptor2 SelectChannelConnector@0.0.0.0:46311, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=381, name=qtp1963244738-381 Acceptor2 SelectChannelConnector@0.0.0.0:46312, state=RUNNABLE, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=351, name=qtp1913461020-351 Acceptor0 SelectChannelConnector@0.0.0.0:46304, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=380, name=qtp1963244738-380 Acceptor1 SelectChannelConnector@0.0.0.0:46312, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=368, name=qtp1557962933-368 Acceptor3 SelectChannelConnector@0.0.0.0:46311, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=391, name=qtp151042765-391 Acceptor2 SelectChannelConnector@0.0.0.0:46316, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=354, name=qtp1913461020-354 Acceptor3 SelectChannelConnector@0.0.0.0:46304, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=366, name=qtp1557962933-366 Acceptor1 SelectChannelConnector@0.0.0.0:46311, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=379, name=qtp1963244738-379 Acceptor0 SelectChannelConnector@0.0.0.0:46312, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=382, name=qtp1963244738-382 Acceptor3 SelectChannelConnector@0.0.0.0:46312, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([398794AF1A5D8DB8]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.component.DistributedSpellCheckComponentTest

Error Message:
36 threads leaked from SUITE scope at org.apache.solr.handler.component.DistributedSpellCheckComponentTest:     1) Thread[id=155, name=qtp1335088353-155 Acceptor1 SelectChannelConnector@0.0.0.0:24364, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=143, name=qtp616953060-143 Acceptor2 SelectChannelConnector@0.0.0.0:63870, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=68, name=qtp956991326-68 Acceptor3 SelectChannelConnector@0.0.0.0:16386, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=141, name=qtp616953060-141 Acceptor0 SelectChannelConnector@0.0.0.0:63870, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=81, name=qtp948186126-81 Acceptor2 SelectChannelConnector@0.0.0.0:14442, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=93, name=qtp28975553-93 Acceptor0 SelectChannelConnector@0.0.0.0:65046, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=38, name=qtp1803748004-38 Acceptor3 SelectChannelConnector@0.0.0.0:40119, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=96, name=qtp28975553-96 Acceptor3 SelectChannelConnector@0.0.0.0:65046, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=36, name=qtp1803748004-36 Acceptor1 SelectChannelConnector@0.0.0.0:40119, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=50, name=qtp1470850187-50 Acceptor1 SelectChannelConnector@0.0.0.0:35702, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=142, name=qtp616953060-142 Acceptor1 SelectChannelConnector@0.0.0.0:63870, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=128, name=qtp1395225393-128 Acceptor1 SelectChannelConnector@0.0.0.0:34750, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=95, name=qtp28975553-95 Acceptor2 SelectChannelConnector@0.0.0.0:65046, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=37, name=qtp1803748004-37 Acceptor2 SelectChannelConnector@0.0.0.0:40119, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=65, name=qtp956991326-65 Acceptor0 SelectChannelConnector@0.0.0.0:16386, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=66, name=qtp956991326-66 Acceptor1 SelectChannelConnector@0.0.0.0:16386, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=154, name=qtp1335088353-154 Acceptor0 SelectChannelConnector@0.0.0.0:24364, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=51, name=qtp1470850187-51 Acceptor2 SelectChannelConnector@0.0.0.0:35702, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=115, name=qtp2033547519-115 Acceptor2 SelectChannelConnector@0.0.0.0:33851, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=94, name=qtp28975553-94 Acceptor1 SelectChannelConnector@0.0.0.0:65046, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   21) Thread[id=79, name=qtp948186126-79 Acceptor0 SelectChannelConnector@0.0.0.0:14442, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   22) Thread[id=80, name=qtp948186126-80 Acceptor1 SelectChannelConnector@0.0.0.0:14442, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   23) Thread[id=52, name=qtp1470850187-52 Acceptor3 SelectChannelConnector@0.0.0.0:35702, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   24) Thread[id=144, name=qtp616953060-144 Acceptor3 SelectChannelConnector@0.0.0.0:63870, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   25) Thread[id=130, name=qtp1395225393-130 Acceptor3 SelectChannelConnector@0.0.0.0:34750, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   26) Thread[id=113, name=qtp2033547519-113 Acceptor0 SelectChannelConnector@0.0.0.0:33851, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   27) Thread[id=127, name=qtp1395225393-127 Acceptor0 SelectChannelConnector@0.0.0.0:34750, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   28) Thread[id=156, name=qtp1335088353-156 Acceptor2 SelectChannelConnector@0.0.0.0:24364, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   29) Thread[id=82, name=qtp948186126-82 Acceptor3 SelectChannelConnector@0.0.0.0:14442, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   30) Thread[id=35, name=qtp1803748004-35 Acceptor0 SelectChannelConnector@0.0.0.0:40119, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   31) Thread[id=116, name=qtp2033547519-116 Acceptor3 SelectChannelConnector@0.0.0.0:33851, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   32) Thread[id=114, name=qtp2033547519-114 Acceptor1 SelectChannelConnector@0.0.0.0:33851, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   33) Thread[id=157, name=qtp1335088353-157 Acceptor3 SelectChannelConnector@0.0.0.0:24364, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   34) Thread[id=49, name=qtp1470850187-49 Acceptor0 SelectChannelConnector@0.0.0.0:35702, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   35) Thread[id=129, name=qtp1395225393-129 Acceptor2 SelectChannelConnector@0.0.0.0:34750, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   36) Thread[id=67, name=qtp956991326-67 Acceptor2 SelectChannelConnector@0.0.0.0:16386, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 36 threads leaked from SUITE scope at org.apache.solr.handler.component.DistributedSpellCheckComponentTest: 
   1) Thread[id=155, name=qtp1335088353-155 Acceptor1 SelectChannelConnector@0.0.0.0:24364, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=143, name=qtp616953060-143 Acceptor2 SelectChannelConnector@0.0.0.0:63870, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=68, name=qtp956991326-68 Acceptor3 SelectChannelConnector@0.0.0.0:16386, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=141, name=qtp616953060-141 Acceptor0 SelectChannelConnector@0.0.0.0:63870, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=81, name=qtp948186126-81 Acceptor2 SelectChannelConnector@0.0.0.0:14442, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=93, name=qtp28975553-93 Acceptor0 SelectChannelConnector@0.0.0.0:65046, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=38, name=qtp1803748004-38 Acceptor3 SelectChannelConnector@0.0.0.0:40119, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=96, name=qtp28975553-96 Acceptor3 SelectChannelConnector@0.0.0.0:65046, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=36, name=qtp1803748004-36 Acceptor1 SelectChannelConnector@0.0.0.0:40119, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=50, name=qtp1470850187-50 Acceptor1 SelectChannelConnector@0.0.0.0:35702, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=142, name=qtp616953060-142 Acceptor1 SelectChannelConnector@0.0.0.0:63870, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=128, name=qtp1395225393-128 Acceptor1 SelectChannelConnector@0.0.0.0:34750, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=95, name=qtp28975553-95 Acceptor2 SelectChannelConnector@0.0.0.0:65046, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=37, name=qtp1803748004-37 Acceptor2 SelectChannelConnector@0.0.0.0:40119, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=65, name=qtp956991326-65 Acceptor0 SelectChannelConnector@0.0.0.0:16386, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=66, name=qtp956991326-66 Acceptor1 SelectChannelConnector@0.0.0.0:16386, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=154, name=qtp1335088353-154 Acceptor0 SelectChannelConnector@0.0.0.0:24364, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=51, name=qtp1470850187-51 Acceptor2 SelectChannelConnector@0.0.0.0:35702, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=115, name=qtp2033547519-115 Acceptor2 SelectChannelConnector@0.0.0.0:33851, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=94, name=qtp28975553-94 Acceptor1 SelectChannelConnector@0.0.0.0:65046, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  21) Thread[id=79, name=qtp948186126-79 Acceptor0 SelectChannelConnector@0.0.0.0:14442, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  22) Thread[id=80, name=qtp948186126-80 Acceptor1 SelectChannelConnector@0.0.0.0:14442, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  23) Thread[id=52, name=qtp1470850187-52 Acceptor3 SelectChannelConnector@0.0.0.0:35702, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  24) Thread[id=144, name=qtp616953060-144 Acceptor3 SelectChannelConnector@0.0.0.0:63870, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  25) Thread[id=130, name=qtp1395225393-130 Acceptor3 SelectChannelConnector@0.0.0.0:34750, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  26) Thread[id=113, name=qtp2033547519-113 Acceptor0 SelectChannelConnector@0.0.0.0:33851, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  27) Thread[id=127, name=qtp1395225393-127 Acceptor0 SelectChannelConnector@0.0.0.0:34750, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  28) Thread[id=156, name=qtp1335088353-156 Acceptor2 SelectChannelConnector@0.0.0.0:24364, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  29) Thread[id=82, name=qtp948186126-82 Acceptor3 SelectChannelConnector@0.0.0.0:14442, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  30) Thread[id=35, name=qtp1803748004-35 Acceptor0 SelectChannelConnector@0.0.0.0:40119, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  31) Thread[id=116, name=qtp2033547519-116 Acceptor3 SelectChannelConnector@0.0.0.0:33851, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  32) Thread[id=114, name=qtp2033547519-114 Acceptor1 SelectChannelConnector@0.0.0.0:33851, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  33) Thread[id=157, name=qtp1335088353-157 Acceptor3 SelectChannelConnector@0.0.0.0:24364, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  34) Thread[id=49, name=qtp1470850187-49 Acceptor0 SelectChannelConnector@0.0.0.0:35702, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  35) Thread[id=129, name=qtp1395225393-129 Acceptor2 SelectChannelConnector@0.0.0.0:34750, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  36) Thread[id=67, name=qtp956991326-67 Acceptor2 SelectChannelConnector@0.0.0.0:16386, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([398794AF1A5D8DB8]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.component.DistributedSpellCheckComponentTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=155, name=qtp1335088353-155 Acceptor1 SelectChannelConnector@0.0.0.0:24364, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=143, name=qtp616953060-143 Acceptor2 SelectChannelConnector@0.0.0.0:63870, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=68, name=qtp956991326-68 Acceptor3 SelectChannelConnector@0.0.0.0:16386, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=141, name=qtp616953060-141 Acceptor0 SelectChannelConnector@0.0.0.0:63870, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=81, name=qtp948186126-81 Acceptor2 SelectChannelConnector@0.0.0.0:14442, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=93, name=qtp28975553-93 Acceptor0 SelectChannelConnector@0.0.0.0:65046, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=38, name=qtp1803748004-38 Acceptor3 SelectChannelConnector@0.0.0.0:40119, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=96, name=qtp28975553-96 Acceptor3 SelectChannelConnector@0.0.0.0:65046, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=36, name=qtp1803748004-36 Acceptor1 SelectChannelConnector@0.0.0.0:40119, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=50, name=qtp1470850187-50 Acceptor1 SelectChannelConnector@0.0.0.0:35702, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=142, name=qtp616953060-142 Acceptor1 SelectChannelConnector@0.0.0.0:63870, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=128, name=qtp1395225393-128 Acceptor1 SelectChannelConnector@0.0.0.0:34750, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=95, name=qtp28975553-95 Acceptor2 SelectChannelConnector@0.0.0.0:65046, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=37, name=qtp1803748004-37 Acceptor2 SelectChannelConnector@0.0.0.0:40119, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=65, name=qtp956991326-65 Acceptor0 SelectChannelConnector@0.0.0.0:16386, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=66, name=qtp956991326-66 Acceptor1 SelectChannelConnector@0.0.0.0:16386, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=154, name=qtp1335088353-154 Acceptor0 SelectChannelConnector@0.0.0.0:24364, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=51, name=qtp1470850187-51 Acceptor2 SelectChannelConnector@0.0.0.0:35702, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=115, name=qtp2033547519-115 Acceptor2 SelectChannelConnector@0.0.0.0:33851, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=79, name=qtp948186126-79 Acceptor0 SelectChannelConnector@0.0.0.0:14442, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   21) Thread[id=94, name=qtp28975553-94 Acceptor1 SelectChannelConnector@0.0.0.0:65046, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   22) Thread[id=80, name=qtp948186126-80 Acceptor1 SelectChannelConnector@0.0.0.0:14442, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   23) Thread[id=52, name=qtp1470850187-52 Acceptor3 SelectChannelConnector@0.0.0.0:35702, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   24) Thread[id=144, name=qtp616953060-144 Acceptor3 SelectChannelConnector@0.0.0.0:63870, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   25) Thread[id=130, name=qtp1395225393-130 Acceptor3 SelectChannelConnector@0.0.0.0:34750, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   26) Thread[id=113, name=qtp2033547519-113 Acceptor0 SelectChannelConnector@0.0.0.0:33851, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   27) Thread[id=127, name=qtp1395225393-127 Acceptor0 SelectChannelConnector@0.0.0.0:34750, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   28) Thread[id=156, name=qtp1335088353-156 Acceptor2 SelectChannelConnector@0.0.0.0:24364, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   29) Thread[id=82, name=qtp948186126-82 Acceptor3 SelectChannelConnector@0.0.0.0:14442, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   30) Thread[id=35, name=qtp1803748004-35 Acceptor0 SelectChannelConnector@0.0.0.0:40119, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   31) Thread[id=116, name=qtp2033547519-116 Acceptor3 SelectChannelConnector@0.0.0.0:33851, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   32) Thread[id=114, name=qtp2033547519-114 Acceptor1 SelectChannelConnector@0.0.0.0:33851, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   33) Thread[id=157, name=qtp1335088353-157 Acceptor3 SelectChannelConnector@0.0.0.0:24364, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   34) Thread[id=49, name=qtp1470850187-49 Acceptor0 SelectChannelConnector@0.0.0.0:35702, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   35) Thread[id=129, name=qtp1395225393-129 Acceptor2 SelectChannelConnector@0.0.0.0:34750, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   36) Thread[id=67, name=qtp956991326-67 Acceptor2 SelectChannelConnector@0.0.0.0:16386, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=155, name=qtp1335088353-155 Acceptor1 SelectChannelConnector@0.0.0.0:24364, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=143, name=qtp616953060-143 Acceptor2 SelectChannelConnector@0.0.0.0:63870, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=68, name=qtp956991326-68 Acceptor3 SelectChannelConnector@0.0.0.0:16386, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=141, name=qtp616953060-141 Acceptor0 SelectChannelConnector@0.0.0.0:63870, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=81, name=qtp948186126-81 Acceptor2 SelectChannelConnector@0.0.0.0:14442, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=93, name=qtp28975553-93 Acceptor0 SelectChannelConnector@0.0.0.0:65046, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=38, name=qtp1803748004-38 Acceptor3 SelectChannelConnector@0.0.0.0:40119, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=96, name=qtp28975553-96 Acceptor3 SelectChannelConnector@0.0.0.0:65046, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=36, name=qtp1803748004-36 Acceptor1 SelectChannelConnector@0.0.0.0:40119, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=50, name=qtp1470850187-50 Acceptor1 SelectChannelConnector@0.0.0.0:35702, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=142, name=qtp616953060-142 Acceptor1 SelectChannelConnector@0.0.0.0:63870, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=128, name=qtp1395225393-128 Acceptor1 SelectChannelConnector@0.0.0.0:34750, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=95, name=qtp28975553-95 Acceptor2 SelectChannelConnector@0.0.0.0:65046, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=37, name=qtp1803748004-37 Acceptor2 SelectChannelConnector@0.0.0.0:40119, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=65, name=qtp956991326-65 Acceptor0 SelectChannelConnector@0.0.0.0:16386, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=66, name=qtp956991326-66 Acceptor1 SelectChannelConnector@0.0.0.0:16386, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=154, name=qtp1335088353-154 Acceptor0 SelectChannelConnector@0.0.0.0:24364, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=51, name=qtp1470850187-51 Acceptor2 SelectChannelConnector@0.0.0.0:35702, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=115, name=qtp2033547519-115 Acceptor2 SelectChannelConnector@0.0.0.0:33851, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=79, name=qtp948186126-79 Acceptor0 SelectChannelConnector@0.0.0.0:14442, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  21) Thread[id=94, name=qtp28975553-94 Acceptor1 SelectChannelConnector@0.0.0.0:65046, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  22) Thread[id=80, name=qtp948186126-80 Acceptor1 SelectChannelConnector@0.0.0.0:14442, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  23) Thread[id=52, name=qtp1470850187-52 Acceptor3 SelectChannelConnector@0.0.0.0:35702, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  24) Thread[id=144, name=qtp616953060-144 Acceptor3 SelectChannelConnector@0.0.0.0:63870, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  25) Thread[id=130, name=qtp1395225393-130 Acceptor3 SelectChannelConnector@0.0.0.0:34750, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  26) Thread[id=113, name=qtp2033547519-113 Acceptor0 SelectChannelConnector@0.0.0.0:33851, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  27) Thread[id=127, name=qtp1395225393-127 Acceptor0 SelectChannelConnector@0.0.0.0:34750, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  28) Thread[id=156, name=qtp1335088353-156 Acceptor2 SelectChannelConnector@0.0.0.0:24364, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  29) Thread[id=82, name=qtp948186126-82 Acceptor3 SelectChannelConnector@0.0.0.0:14442, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  30) Thread[id=35, name=qtp1803748004-35 Acceptor0 SelectChannelConnector@0.0.0.0:40119, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  31) Thread[id=116, name=qtp2033547519-116 Acceptor3 SelectChannelConnector@0.0.0.0:33851, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  32) Thread[id=114, name=qtp2033547519-114 Acceptor1 SelectChannelConnector@0.0.0.0:33851, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  33) Thread[id=157, name=qtp1335088353-157 Acceptor3 SelectChannelConnector@0.0.0.0:24364, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  34) Thread[id=49, name=qtp1470850187-49 Acceptor0 SelectChannelConnector@0.0.0.0:35702, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  35) Thread[id=129, name=qtp1395225393-129 Acceptor2 SelectChannelConnector@0.0.0.0:34750, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  36) Thread[id=67, name=qtp956991326-67 Acceptor2 SelectChannelConnector@0.0.0.0:16386, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([398794AF1A5D8DB8]:0)




Build Log:
[...truncated 8342 lines...]
[junit4:junit4] Suite: org.apache.solr.handler.component.DistributedSpellCheckComponentTest
[junit4:junit4]   2> 4 T28 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> 610 T28 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 854 T28 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:40119
[junit4:junit4]   2> 903 T28 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 903 T28 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 904 T28 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 952 T28 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 952 T28 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 953 T28 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 953 T28 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 954 T28 oasc.CoreContainer.<init> New CoreContainer 577133894
[junit4:junit4]   2> 955 T28 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 955 T28 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 991 T28 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 1034 T28 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 1035 T28 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 1036 T28 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 1037 T28 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 1113 T28 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 1185 T28 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1186 T28 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 1227 T28 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1928 T28 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1941 T28 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1945 T28 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1962 T28 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1967 T28 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1971 T28 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1974 T28 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1976 T28 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.handler.component.DistributedSpellCheckComponentTest-1351832296783/control/data/
[junit4:junit4]   2> 1977 T28 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@d424b65
[junit4:junit4]   2> 2004 T28 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.handler.component.DistributedSpellCheckComponentTest-1351832296783/control/data/index/
[junit4:junit4]   2> 2005 T28 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.handler.component.DistributedSpellCheckComponentTest-1351832296783/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2006 T28 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.handler.component.DistributedSpellCheckComponentTest-1351832296783/control/data/index forceNew:false
[junit4:junit4]   2> 2016 T28 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.handler.component.DistributedSpellCheckComponentTest-1351832296783/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3603845b; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 2017 T28 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 2018 T28 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 2020 T28 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2021 T28 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 2022 T28 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2023 T28 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 2024 T28 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2024 T28 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2025 T28 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2026 T28 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2027 T28 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2028 T28 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 2030 T28 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 2030 T28 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2031 T28 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 2032 T28 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 2032 T28 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 2033 T28 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2034 T28 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2035 T28 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2037 T28 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2038 T28 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2039 T28 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2040 T28 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2041 T28 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2042 T28 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 2043 T28 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 2052 T28 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2055 T28 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 2057 T28 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [THROW.ERROR.ON.ADD.UPDATEPROCESSOR.JS, SOLRCONFIG-HIGHLIGHT.XML, SCHEMA-REQUIRED-FIELDS.XML, BAD-SCHEMA-DUP-DYNAMICFIELD.XML, SCHEMA-REPLICATION2.XML, TRIVIAL.UPDATEPROCESSOR1.JS, SCHEMA-MINIMAL.XML, SOLRCONFIG-CACHING.XML, SOLRCONFIG-REPEATER.XML, BAD-SCHEMA-NONTEXT-ANALYZER.XML, SOLRCONFIG-MERGEPOLICY.XML, CURRENCY.XML, SOLRCONFIG-TLOG.XML, SOLRCONFIG-MASTER.XML, BAD-SCHEMA-UNIQUEKEY-MULTIVALUED.XML, SCHEMA11.XML, SOLRCONFIG-BASIC.XML, DA_COMPOUNDDICTIONARY.TXT, MISSLEADING.EXTENSION.UPDATEPROCESSOR.JS.TXT, SCHEMA-COPYFIELD-TEST.XML, SOLRCONFIG-SLAVE.XML, ELEVATE.XML, SOLRCONFIG-PROPINJECT-INDEXDEFAULT.XML, TRIVIAL.UPDATEPROCESSOR0.JS, BAD-SOLRCONFIG-BOGUS-SCRIPTENGINE-NAME.XML, SCHEMA-CHARFILTERS.XML, SCHEMA-IB.XML, SOLRCONFIG-QUERYSENDER.XML, SOLRCONFIG-SNIPPET-PROCESSOR.XML, SCHEMA-REPLICATION1.XML, DA_UTF8.XML, CONDITIONAL.UPDATEPROCESSOR.JS, MISSING.FUNCTIONS.UPDATEPROCESSOR.JS, HYPHENATION.DTD, SOLRCONFIG-ENABLEPLUGIN.XML, STEMDICT.TXT, SCHEMA-PHRASESUGGEST.XML, HUNSPELL-TEST.AFF, SCHEMA-SNIPPET-TYPE.XML, STOPTYPES-1.TXT, STOPWORDSWRONGENCODING.TXT, SCHEMA-NUMERIC.XML, SOLRCONFIG-TRANSFORMERS.XML, SOLRCONFIG-PROPINJECT.XML, BAD-SCHEMA-NOT-INDEXED-BUT-TF.XML, SOLRCONFIG-SIMPLELOCK.XML, WDFTYPES.TXT, STOPTYPES-2.TXT, SCHEMA-REVERSED.XML, BAD-SCHEMA-CURRENCY-FT-MULTIVALUED.XML, SOLRCONFIG-SPELLCHECKCOMPONENT.XML, SCHEMA-DFR.XML, SOLRCONFIG-PHRASESUGGEST.XML, BAD-SCHEMA-CURRENCY-MULTIVALUED.XML, BAD-SCHEMA-NOT-INDEXED-BUT-POS.XML, KEEP-1.TXT, OPEN-EXCHANGE-RATES.JSON, SCHEMA-SPATIAL.XML, STOPWITHBOM.TXT, SOLRCONFIG-SPELLCHECKER.XML, SCHEMA-BINARYFIELD.XML, SOLRCONFIG-UPDATE-PROCESSOR-CHAINS.XML, BAD-SCHEMA-OMIT-TF-BUT-NOT-POS.XML, BAD-SCHEMA-DUP-FIELDTYPE.XML, SCHEMA-XINCLUDE.XML, SOLRCONFIG-MASTER1.XML, SYNONYMS.TXT, BAD-SCHEMA-CURRENCY-DYNAMIC-MULTIVALUED.XML, SCHEMA_CODEC.XML, SCHEMA.XML, SOLRCONFIG-SOLR-749.XML, SOLRCONFIG-MASTER1-KEEPONEBACKUP.XML, SOLRCONFIG-FUNCTIONQUERY.XML, STOP-2.TXT, SCHEMA-LMDIRICHLET.XML, SOLRCONFIG-TERMINDEX.XML, SOLRCONFIG-ELEVATE.XML, SCHEMA-FOLDING.XML, STOPWORDS.TXT, SCHEMA-STOP-KEEP.XML, BAD-SCHEMA-NOT-INDEXED-BUT-NORMS.XML, SOLRCONFIG-SOLCOREPROPERTIES.XML, STOP-1.TXT, SOLRCONFIG-MASTER2.XML, SCHEMA-SPELLCHECKER.XML, SOLRCONFIG-RESPONSE-LOG-COMPONENT.XML, SOLRCONFIG-LAZYWRITER.XML, SCHEMA-LUCENEMATCHVERSION.XML, BAD-MP-SOLRCONFIG.XML, FRENCHARTICLES.TXT, SCHEMA15.XML, SOLRCONFIG-REQHANDLER.INCL, SCHEMASURROUND.XML, SCHEMA-COLLATEFILTER.XML, SOLRCONFIG-MASTER3.XML, HUNSPELL-TEST.DIC, BAD-SCHEMA-CODEC-GLOBAL-VS-FT-MISMATCH.XML, SOLRCONFIG-XINCLUDE.XML, SOLRCONFIG-SLAVE1.XML, SOLRCONFIG-DELPOLICY1.XML, SCHEMA-SIM.XML, SCHEMA-SNIPPET-FIELD.XML, SCHEMA-COLLATE.XML, STOP-SNOWBALL.TXT, BAD-SCHEMA-SIM-GLOBAL-VS-FT-MISMATCH.XML, PROTWORDS.TXT, SCHEMA-TRIE.XML, SOLRCONFIG_CODEC.XML, BAD-SOLRCONFIG-INVALID-SCRIPTFILE.XML, JASUGGEST.TXT, SCHEMA-TFIDF.XML, SOLRCONFIG-SCRIPT-UPDATEPROCESSOR.XML, SCHEMA-LMJELINEKMERCER.XML, PHRASESUGGEST.TXT, BAD-SOLRCONFIG-MISSING-SCRIPTFILE.XML, SOLRCONFIG-BASIC-LUCENEVERSION31.XML, BAD-SCHEMA-UNIQUEKEY-USES-DEFAULT.XML, OLD_SYNONYMS.TXT, SOLRCONFIG-DELPOLICY2.XML, SOLRCONFIG-NATIVELOCK.XML, XSLT, BAD-SCHEMA-DUP-FIELD.XML, SOLRCONFIG-NOCACHE.XML, SCHEMA-BM25.XML, ADDFIELDS.UPDATEPROCESSOR.JS, SOLRCONFIG-QUERYSENDER-NOQUERY.XML, SOLRCONFIG-ALTDIRECTORY.XML, COMPOUNDDICTIONARY.TXT, SOLRCONFIG-INDEXCONFIG.XML, SOLRCONFIG_PERF.XML, SCHEMA-NOT-REQUIRED-UNIQUE-KEY.XML, BAD-SCHEMA-ANALYZER-CLASS-AND-NESTED.XML, KEEP-2.TXT, BAD-SCHEMA-UNIQUEKEY-IS-COPYFIELD-DEST.XML, SCHEMA12.XML, MAPPING-ISOLATIN1ACCENT.TXT, BAD_SOLRCONFIG.XML, BAD-SCHEMA-EXTERNAL-FILEFIELD.XML]
[junit4:junit4]   2> 2062 T28 oass.SolrIndexSearcher.<init> Opening Searcher@44f3a157 main
[junit4:junit4]   2> 2062 T28 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2063 T28 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2064 T28 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 2075 T28 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 2119 T28 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 2119 T28 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 2120 T28 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 2120 T28 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 2121 T28 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 2121 T28 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 2122 T28 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 2122 T28 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 2123 T28 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 2123 T28 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 2149 T43 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@44f3a157 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2158 T28 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 2159 T28 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 2160 T28 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 2206 T28 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 2224 T28 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 2226 T28 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:35702
[junit4:junit4]   2> 2227 T28 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2228 T28 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 2228 T28 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2257 T28 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 2258 T28 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2258 T28 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 2259 T28 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 2259 T28 oasc.CoreContainer.<init> New CoreContainer 2064783193
[junit4:junit4]   2> 2260 T28 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2260 T28 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2293 T28 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 2310 T28 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 2311 T28 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 2312 T28 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 2313 T28 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 2409 T28 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 2484 T28 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 2485 T28 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 2516 T28 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 3058 T28 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 3071 T28 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 3074 T28 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 3091 T28 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 3096 T28 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 3100 T28 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 3103 T28 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 3105 T28 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.handler.component.DistributedSpellCheckComponentTest-1351832296783/shard0/data/
[junit4:junit4]   2> 3105 T28 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@d424b65
[junit4:junit4]   2> 3107 T28 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.handler.component.DistributedSpellCheckComponentTest-1351832296783/shard0/data/index/
[junit4:junit4]   2> 3108 T28 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.handler.component.DistributedSpellCheckComponentTest-1351832296783/shard0/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 3109 T28 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.handler.component.DistributedSpellCheckComponentTest-1351832296783/shard0/data/index forceNew:false
[junit4:junit4]   2> 3266 T28 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.handler.component.DistributedSpellCheckComponentTest-1351832296783/shard0/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@463162dc; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 3267 T28 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 3268 T28 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 3271 T28 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 3272 T28 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 3272 T28 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 3273 T28 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 3274 T28 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 3274 T28 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 3275 T28 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 3276 T28 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 3277 T28 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 3278 T28 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 3279 T28 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 3280 T28 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 3281 T28 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 3282 T28 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 3282 T28 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 3284 T28 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 3285 T28 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3286 T28 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3287 T28 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3288 T28 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3289 T28 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3290 T28 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3291 T28 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3292 T28 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 3294 T28 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 3302 T28 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 3306 T28 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 3308 T28 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [THROW.ERROR.ON.ADD.UPDATEPROCESSOR.JS, SOLRCONFIG-HIGHLIGHT.XML, SCHEMA-REQUIRED-FIELDS.XML, BAD-SCHEMA-DUP-DYNAMICFIELD.XML, SCHEMA-REPLICATION2.XML, TRIVIAL.UPDATEPROCESSOR1.JS, SCHEMA-MINIMAL.XML, SOLRCONFIG-CACHING.XML, SOLRCONFIG-REPEATER.XML, BAD-SCHEMA-NONTEXT-ANALYZER.XML, SOLRCONFIG-MERGEPOLICY.XML, CURRENCY.XML, SOLRCONFIG-TLOG.XML, SOLRCONFIG-MASTER.XML, BAD-SCHEMA-UNIQUEKEY-MULTIVALUED.XML, SCHEMA11.XML, SOLRCONFIG-BASIC.XML, DA_COMPOUNDDICTIONARY.TXT, MISSLEADING.EXTENSION.UPDATEPROCESSOR.JS.TXT, SCHEMA-COPYFIELD-TEST.XML, SOLRCONFIG-SLAVE.XML, ELEVATE.XML, SOLRCONFIG-PROPINJECT-INDEXDEFAULT.XML, TRIVIAL.UPDATEPROCESSOR0.JS, BAD-SOLRCONFIG-BOGUS-SCRIPTENGINE-NAME.XML, SCHEMA-CHARFILTERS.XML, SCHEMA-IB.XML, SOLRCONFIG-QUERYSENDER.XML, SOLRCONFIG-SNIPPET-PROCESSOR.XML, SCHEMA-REPLICATION1.XML, DA_UTF8.XML, CONDITIONAL.UPDATEPROCESSOR.JS, MISSING.FUNCTIONS.UPDATEPROCESSOR.JS, HYPHENATION.DTD, SOLRCONFIG-ENABLEPLUGIN.XML, STEMDICT.TXT, SCHEMA-PHRASESUGGEST.XML, HUNSPELL-TEST.AFF, SCHEMA-SNIPPET-TYPE.XML, STOPTYPES-1.TXT, STOPWORDSWRONGENCODING.TXT, SCHEMA-NUMERIC.XML, SOLRCONFIG-TRANSFORMERS.XML, SOLRCONFIG-PROPINJECT.XML, BAD-SCHEMA-NOT-INDEXED-BUT-TF.XML, SOLRCONFIG-SIMPLELOCK.XML, WDFTYPES.TXT, STOPTYPES-2.TXT, SCHEMA-REVERSED.XML, BAD-SCHEMA-CURRENCY-FT-MULTIVALUED.XML, SOLRCONFIG-SPELLCHECKCOMPONENT.XML, SCHEMA-DFR.XML, SOLRCONFIG-PHRASESUGGEST.XML, BAD-SCHEMA-CURRENCY-MULTIVALUED.XML, BAD-SCHEMA-NOT-INDEXED-BUT-POS.XML, KEEP-1.TXT, OPEN-EXCHANGE-RATES.JSON, SCHEMA-SPATIAL.XML, STOPWITHBOM.TXT, SOLRCONFIG-SPELLCHECKER.XML, SCHEMA-BINARYFIELD.XML, SOLRCONFIG-UPDATE-PROCESSOR-CHAINS.XML, BAD-SCHEMA-OMIT-TF-BUT-NOT-POS.XML, BAD-SCHEMA-DUP-FIELDTYPE.XML, SCHEMA-XINCLUDE.XML, SOLRCONFIG-MASTER1.XML, SYNONYMS.TXT, BAD-SCHEMA-CURRENCY-DYNAMIC-MULTIVALUED.XML, SCHEMA_CODEC.XML, SCHEMA.XML, SOLRCONFIG-SOLR-749.XML, SOLRCONFIG-MASTER1-KEEPONEBACKUP.XML, SOLRCONFIG-FUNCTIONQUERY.XML, STOP-2.TXT, SCHEMA-LMDIRICHLET.XML, SOLRCONFIG-TERMINDEX.XML, SOLRCONFIG-ELEVATE.XML, SCHEMA-FOLDING.XML, STOPWORDS.TXT, SCHEMA-STOP-KEEP.XML, BAD-SCHEMA-NOT-INDEXED-BUT-NORMS.XML, SOLRCONFIG-SOLCOREPROPERTIES.XML, STOP-1.TXT, SOLRCONFIG-MASTER2.XML, SCHEMA-SPELLCHECKER.XML, SOLRCONFIG-RESPONSE-LOG-COMPONENT.XML, SOLRCONFIG-LAZYWRITER.XML, SCHEMA-LUCENEMATCHVERSION.XML, BAD-MP-SOLRCONFIG.XML, FRENCHARTICLES.TXT, SCHEMA15.XML, SOLRCONFIG-REQHANDLER.INCL, SCHEMASURROUND.XML, SCHEMA-COLLATEFILTER.XML, SOLRCONFIG-MASTER3.XML, HUNSPELL-TEST.DIC, BAD-SCHEMA-CODEC-GLOBAL-VS-FT-MISMATCH.XML, SOLRCONFIG-XINCLUDE.XML, SOLRCONFIG-SLAVE1.XML, SOLRCONFIG-DELPOLICY1.XML, SCHEMA-SIM.XML, SCHEMA-SNIPPET-FIELD.XML, SCHEMA-COLLATE.XML, STOP-SNOWBALL.TXT, BAD-SCHEMA-SIM-GLOBAL-VS-FT-MISMATCH.XML, PROTWORDS.TXT, SCHEMA-TRIE.XML, SOLRCONFIG_CODEC.XML, BAD-SOLRCONFIG-INVALID-SCRIPTFILE.XML, JASUGGEST.TXT, SCHEMA-TFIDF.XML, SOLRCONFIG-SCRIPT-UPDATEPROCESSOR.XML, SCHEMA-LMJELINEKMERCER.XML, PHRASESUGGEST.TXT, BAD-SOLRCONFIG-MISSING-SCRIPTFILE.XML, SOLRCONFIG-BASIC-LUCENEVERSION31.XML, BAD-SCHEMA-UNIQUEKEY-USES-DEFAULT.XML, OLD_SYNONYMS.TXT, SOLRCONFIG-DELPOLICY2.XML, SOLRCONFIG-NATIVELOCK.XML, XSLT, BAD-SCHEMA-DUP-FIELD.XML, SOLRCONFIG-NOCACHE.XML, SCHEMA-BM25.XML, ADDFIELDS.UPDATEPROCESSOR.JS, SOLRCONFIG-QUERYSENDER-NOQUERY.XML, SOLRCONFIG-ALTDIRECTORY.XML, COMPOUNDDICTIONARY.TXT, SOLRCONFIG-INDEXCONFIG.XML, SOLRCONFIG_PERF.XML, SCHEMA-NOT-REQUIRED-UNIQUE-KEY.XML, BAD-SCHEMA-ANALYZER-CLASS-AND-NESTED.XML, KEEP-2.TXT, BAD-SCHEMA-UNIQUEKEY-IS-COPYFIELD-DEST.XML, SCHEMA12.XML, MAPPING-ISOLATIN1ACCENT.TXT, BAD_SOLRCONFIG.XML, BAD-SCHEMA-EXTERNAL-FILEFIELD.XML]
[junit4:junit4]   2> 3313 T28 oass.SolrIndexSearcher.<init> Opening Searcher@199bdabd main
[junit4:junit4]   2> 3314 T28 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 3314 T28 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 3316 T28 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 3333 T28 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 3384 T28 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 3384 T28 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 3385 T28 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 3385 T28 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 3385 T28 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 3386 T28 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 3386 T28 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 3387 T28 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 3387 T28 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 3388 T28 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 3410 T57 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@199bdabd main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 3420 T28 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 3421 T28 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 3421 T28 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3422 T28 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C0 name=collection1 org.apache.solr.core.SolrCore@1e345a8a
[junit4:junit4]   2> 4976 T41 C0 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4:junit4]   2> 4986 T41 C0 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.handler.component.DistributedSpellCheckComponentTest-1351832296783/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3603845b; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 4987 T41 C0 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 4993 T41 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 589
[junit4:junit4]   2> ASYNC  NEW_CORE C1 name=collection1 org.apache.solr.core.SolrCore@129ebc45
[junit4:junit4]   2> 5175 T56 C1 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4:junit4]   2> 5179 T56 C1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.handler.component.DistributedSpellCheckComponentTest-1351832296783/shard0/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@463162dc; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5179 T56 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 5180 T56 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 6
[junit4:junit4]   2> 5435 T42 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1]} 0 243
[junit4:junit4]   2> 5443 T55 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1]} 0 4
[junit4:junit4]   2> 5451 T41 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[2]} 0 3
[junit4:junit4]   2> 5457 T56 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[2]} 0 2
[junit4:junit4]   2> 5464 T40 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[3]} 0 2
[junit4:junit4]   2> 5471 T54 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[3]} 0 2
[junit4:junit4]   2> 5476 T42 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[4]} 0 2
[junit4:junit4]   2> 5482 T55 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[4]} 0 2
[junit4:junit4]   2> 5488 T41 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[5]} 0 1
[junit4:junit4]   2> 5493 T56 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[5]} 0 1
[junit4:junit4]   2> 5498 T42 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[6]} 0 2
[junit4:junit4]   2> 5503 T54 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[6]} 0 2
[junit4:junit4]   2> 5508 T40 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[7]} 0 1
[junit4:junit4]   2> 5516 T55 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[7]} 0 2
[junit4:junit4]   2> 5522 T41 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[8]} 0 2
[junit4:junit4]   2> 5527 T56 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[8]} 0 1
[junit4:junit4]   2> 5534 T42 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[9]} 0 2
[junit4:junit4]   2> 5541 T54 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[9]} 0 3
[junit4:junit4]   2> 5981 T40 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[10]} 0 436
[junit4:junit4]   2> 6006 T56 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[10]} 0 21
[junit4:junit4]   2> 6013 T42 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[12]} 0 2
[junit4:junit4]   2> 6020 T54 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[12]} 0 3
[junit4:junit4]   2> 6027 T41 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[13]} 0 2
[junit4:junit4]   2> 6034 T55 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[13]} 0 3
[junit4:junit4]   2> 6039 T40 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[14]} 0 1
[junit4:junit4]   2> 6045 T56 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[14]} 0 2
[junit4:junit4]   2> 6051 T42 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[15]} 0 2
[junit4:junit4]   2> 6058 T54 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[15]} 0 2
[junit4:junit4]   2> 6065 T41 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[16]} 0 2
[junit4:junit4]   2> 6072 T55 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[16]} 0 2
[junit4:junit4]   2> 6078 T42 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[17]} 0 2
[junit4:junit4]   2> 6085 T56 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[17]} 0 2
[junit4:junit4]   2> 6092 T41 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[18]} 0 2
[junit4:junit4]   2> 6099 T54 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[18]} 0 2
[junit4:junit4]   2> 6105 T40 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[19]} 0 2
[junit4:junit4]   2> 6111 T55 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[19]} 0 2
[junit4:junit4]   2> 6118 T42 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[20]} 0 2
[junit4:junit4]   2> 6125 T56 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[20]} 0 2
[junit4:junit4]   2> 6150 T41 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[21]} 0 20
[junit4:junit4]   2> 6174 T54 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[21]} 0 19
[junit4:junit4]   2> 6183 T40 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[22]} 0 4
[junit4:junit4]   2> 6191 T55 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[22]} 0 4
[junit4:junit4]   2> 6198 T42 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[23]} 0 2
[junit4:junit4]   2> 6204 T56 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[23]} 0 2
[junit4:junit4]   2> 6211 T41 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[24]} 0 2
[junit4:junit4]   2> 6218 T54 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[24]} 0 2
[junit4:junit4]   2> 6225 T40 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[25]} 0 2
[junit4:junit4]   2> 6231 T55 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[25]} 0 2
[junit4:junit4]   2> 6270 T42 C0 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 6867 T42 C0 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.handler.component.DistributedSpellCheckComponentTest-1351832296783/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3603845b; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.handler.component.DistributedSpellCheckComponentTest-1351832296783/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3603845b; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[_2_Lucene41_0.pos, _1_Lucene41_0.doc, _2_Lucene41_0.tim, _0_Lucene41_0.pos, _1_MockFixedIntBlock_0.pyl, _0_MockFixedIntBlock_0.tib, _2_Lucene41_0.tip, _1.fnm, _2_MockFixedIntBlock_0.tii, _2_MockFixedIntBlock_0.tib, _2_MockFixedIntBlock_0.pyl, _0_MockFixedIntBlock_0.tii, _2_MockFixedIntBlock_0.frq, _0_Lucene41_0.tim, _0_Lucene41_0.tip, _0.fnm, _1_MockFixedIntBlock_0.frq, _0_Memory_0.ram, _2.fdt, _0_MockFixedIntBlock_0.pos, _2_Lucene41_0.doc, _2.fdx, _1.fdx, _1_Lucene41_0.pos, _1_MockFixedIntBlock_0.skp, _2_MockFixedIntBlock_0.pos, _0_Lucene41_0.doc, _1.fdt, _2_MockFixedIntBlock_0.doc, _1_MockFixedIntBlock_0.pos, _2.si, _1_MockFixedIntBlock_0.doc, _2_MockFixedIntBlock_0.skp, _1_Memory_0.ram, _2_nrm.cfe, _0_MockFixedIntBlock_0.pyl, _0_MockFixedIntBlock_0.skp, _2_Memory_0.ram, _2_nrm.cfs, _0.si, _1_MockFixedIntBlock_0.tii, _1_nrm.cfs, _1_Lucene41_0.tim, _1_Lucene41_0.tip, _0_nrm.cfs, _1_nrm.cfe, _2.fnm, _0_MockFixedIntBlock_0.frq, _0_MockFixedIntBlock_0.doc, _1.si, _0_nrm.cfe, segments_2, _1_MockFixedIntBlock_0.tib, _0.fdx, _0.fdt]
[junit4:junit4]   2> 6868 T42 C0 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 6921 T42 C0 oass.SolrIndexSearcher.<init> Opening Searcher@1aa42f23 main
[junit4:junit4]   2> 6926 T42 C0 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 6938 T43 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1aa42f23 main{StandardDirectoryReader(segments_2:8 _0(4.1):C10 _1(4.1):C10 _2(4.1):C4)}
[junit4:junit4]   2> 6940 T42 C0 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 674
[junit4:junit4]   2> 6945 T55 C1 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 7106 T55 C1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.handler.component.DistributedSpellCheckComponentTest-1351832296783/shard0/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@463162dc; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.handler.component.DistributedSpellCheckComponentTest-1351832296783/shard0/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@463162dc; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[_2_Lucene41_0.pos, _1_Lucene41_0.doc, _2_Lucene41_0.tim, _0_Lucene41_0.pos, _1_MockFixedIntBlock_0.pyl, _0_MockFixedIntBlock_0.tib, _2_Lucene41_0.tip, _1.fnm, _2_MockFixedIntBlock_0.tii, _2_MockFixedIntBlock_0.tib, _2_MockFixedIntBlock_0.pyl, _0_MockFixedIntBlock_0.tii, _2_MockFixedIntBlock_0.frq, _0_Lucene41_0.tim, _0_Lucene41_0.tip, _0.fnm, _1_MockFixedIntBlock_0.frq, _0_Memory_0.ram, _2.fdt, _0_MockFixedIntBlock_0.pos, _2_Lucene41_0.doc, _2.fdx, _1.fdx, _1_Lucene41_0.pos, _1_MockFixedIntBlock_0.skp, _2_MockFixedIntBlock_0.pos, _0_Lucene41_0.doc, _1.fdt, _2_MockFixedIntBlock_0.doc, _1_MockFixedIntBlock_0.pos, _2.si, _1_MockFixedIntBlock_0.doc, _2_MockFixedIntBlock_0.skp, _1_Memory_0.ram, _2_nrm.cfe, _0_MockFixedIntBlock_0.pyl, _0_MockFixedIntBlock_0.skp, _2_Memory_0.ram, _2_nrm.cfs, _0.si, _1_MockFixedIntBlock_0.tii, _1_nrm.cfs, _1_Lucene41_0.tim, _1_Lucene41_0.tip, _0_nrm.cfs, _1_nrm.cfe, _2.fnm, _0_MockFixedIntBlock_0.frq, _0_MockFixedIntBlock_0.doc, _1.si, _0_nrm.cfe, segments_2, _1_MockFixedIntBlock_0.tib, _0.fdx, _0.fdt]
[junit4:junit4]   2> 7107 T55 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 7152 T55 C1 oass.SolrIndexSearcher.<init> Opening Searcher@560cb746 main
[junit4:junit4]   2> 7153 T55 C1 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 7160 T57 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@560cb746 main{StandardDirectoryReader(segments_2:8 _0(4.1):C10 _1(4.1):C10 _2(4.1):C4)}
[junit4:junit4]   2> 7162 T55 C1 UPDATE [collection1] webapp=/solr path=/update p

[...truncated too long message...]

>         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    7) Thread[id=367, name=qtp1557962933-367 Acceptor2 SelectChannelConnector@0.0.0.0:46311, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    8) Thread[id=381, name=qtp1963244738-381 Acceptor2 SelectChannelConnector@0.0.0.0:46312, state=RUNNABLE, group=TGRP-DistributedQueryElevationComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    9) Thread[id=351, name=qtp1913461020-351 Acceptor0 SelectChannelConnector@0.0.0.0:46304, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   10) Thread[id=380, name=qtp1963244738-380 Acceptor1 SelectChannelConnector@0.0.0.0:46312, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   11) Thread[id=368, name=qtp1557962933-368 Acceptor3 SelectChannelConnector@0.0.0.0:46311, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   12) Thread[id=391, name=qtp151042765-391 Acceptor2 SelectChannelConnector@0.0.0.0:46316, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   13) Thread[id=354, name=qtp1913461020-354 Acceptor3 SelectChannelConnector@0.0.0.0:46304, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   14) Thread[id=366, name=qtp1557962933-366 Acceptor1 SelectChannelConnector@0.0.0.0:46311, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   15) Thread[id=379, name=qtp1963244738-379 Acceptor0 SelectChannelConnector@0.0.0.0:46312, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   16) Thread[id=382, name=qtp1963244738-382 Acceptor3 SelectChannelConnector@0.0.0.0:46312, state=BLOCKED, group=TGRP-DistributedQueryElevationComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([398794AF1A5D8DB8]:0)
[junit4:junit4] Completed on J1 in 28.60s, 1 test, 2 errors <<< FAILURES!

[...truncated 83 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:335: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/module-build.xml:61: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:1163: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:827: There were test failures: 233 suites, 965 tests, 4 suite-level errors, 699 ignored

Total time: 20 minutes 16 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure

[JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 605 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.

Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/605/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.servlet.CacheHeaderTest

Error Message:
4 threads leaked from SUITE scope at org.apache.solr.servlet.CacheHeaderTest:     1) Thread[id=321, name=qtp77073360-321 Acceptor3 SelectChannelConnector@0.0.0.0:23627, state=BLOCKED, group=TGRP-CacheHeaderTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=318, name=qtp77073360-318 Acceptor0 SelectChannelConnector@0.0.0.0:23627, state=BLOCKED, group=TGRP-CacheHeaderTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=319, name=qtp77073360-319 Acceptor1 SelectChannelConnector@0.0.0.0:23627, state=BLOCKED, group=TGRP-CacheHeaderTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=320, name=qtp77073360-320 Acceptor2 SelectChannelConnector@0.0.0.0:23627, state=RUNNABLE, group=TGRP-CacheHeaderTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.servlet.CacheHeaderTest: 
   1) Thread[id=321, name=qtp77073360-321 Acceptor3 SelectChannelConnector@0.0.0.0:23627, state=BLOCKED, group=TGRP-CacheHeaderTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=318, name=qtp77073360-318 Acceptor0 SelectChannelConnector@0.0.0.0:23627, state=BLOCKED, group=TGRP-CacheHeaderTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=319, name=qtp77073360-319 Acceptor1 SelectChannelConnector@0.0.0.0:23627, state=BLOCKED, group=TGRP-CacheHeaderTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=320, name=qtp77073360-320 Acceptor2 SelectChannelConnector@0.0.0.0:23627, state=RUNNABLE, group=TGRP-CacheHeaderTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([9F31DB9569802843]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.servlet.CacheHeaderTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=321, name=qtp77073360-321 Acceptor3 SelectChannelConnector@0.0.0.0:23627, state=BLOCKED, group=TGRP-CacheHeaderTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=318, name=qtp77073360-318 Acceptor0 SelectChannelConnector@0.0.0.0:23627, state=BLOCKED, group=TGRP-CacheHeaderTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=319, name=qtp77073360-319 Acceptor1 SelectChannelConnector@0.0.0.0:23627, state=BLOCKED, group=TGRP-CacheHeaderTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=320, name=qtp77073360-320 Acceptor2 SelectChannelConnector@0.0.0.0:23627, state=RUNNABLE, group=TGRP-CacheHeaderTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=321, name=qtp77073360-321 Acceptor3 SelectChannelConnector@0.0.0.0:23627, state=BLOCKED, group=TGRP-CacheHeaderTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=318, name=qtp77073360-318 Acceptor0 SelectChannelConnector@0.0.0.0:23627, state=BLOCKED, group=TGRP-CacheHeaderTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=319, name=qtp77073360-319 Acceptor1 SelectChannelConnector@0.0.0.0:23627, state=BLOCKED, group=TGRP-CacheHeaderTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=320, name=qtp77073360-320 Acceptor2 SelectChannelConnector@0.0.0.0:23627, state=RUNNABLE, group=TGRP-CacheHeaderTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([9F31DB9569802843]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.update.PeerSyncTest

Error Message:
16 threads leaked from SUITE scope at org.apache.solr.update.PeerSyncTest:     1) Thread[id=74, name=qtp1680555153-74 Acceptor2 SelectChannelConnector@0.0.0.0:46108, state=BLOCKED, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=33, name=qtp1128420885-33 Acceptor2 SelectChannelConnector@0.0.0.0:20160, state=BLOCKED, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=72, name=qtp1680555153-72 Acceptor0 SelectChannelConnector@0.0.0.0:46108, state=BLOCKED, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=32, name=qtp1128420885-32 Acceptor1 SelectChannelConnector@0.0.0.0:20160, state=RUNNABLE, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=31, name=qtp1128420885-31 Acceptor0 SelectChannelConnector@0.0.0.0:20160, state=BLOCKED, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=61, name=qtp1209461881-61 Acceptor2 SelectChannelConnector@0.0.0.0:32925, state=BLOCKED, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=59, name=qtp1209461881-59 Acceptor0 SelectChannelConnector@0.0.0.0:32925, state=BLOCKED, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=47, name=qtp2106422771-47 Acceptor2 SelectChannelConnector@0.0.0.0:44729, state=BLOCKED, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=46, name=qtp2106422771-46 Acceptor1 SelectChannelConnector@0.0.0.0:44729, state=BLOCKED, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=62, name=qtp1209461881-62 Acceptor3 SelectChannelConnector@0.0.0.0:32925, state=RUNNABLE, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=60, name=qtp1209461881-60 Acceptor1 SelectChannelConnector@0.0.0.0:32925, state=BLOCKED, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=34, name=qtp1128420885-34 Acceptor3 SelectChannelConnector@0.0.0.0:20160, state=BLOCKED, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=73, name=qtp1680555153-73 Acceptor1 SelectChannelConnector@0.0.0.0:46108, state=BLOCKED, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=75, name=qtp1680555153-75 Acceptor3 SelectChannelConnector@0.0.0.0:46108, state=RUNNABLE, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=45, name=qtp2106422771-45 Acceptor0 SelectChannelConnector@0.0.0.0:44729, state=BLOCKED, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=48, name=qtp2106422771-48 Acceptor3 SelectChannelConnector@0.0.0.0:44729, state=RUNNABLE, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 16 threads leaked from SUITE scope at org.apache.solr.update.PeerSyncTest: 
   1) Thread[id=74, name=qtp1680555153-74 Acceptor2 SelectChannelConnector@0.0.0.0:46108, state=BLOCKED, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=33, name=qtp1128420885-33 Acceptor2 SelectChannelConnector@0.0.0.0:20160, state=BLOCKED, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=72, name=qtp1680555153-72 Acceptor0 SelectChannelConnector@0.0.0.0:46108, state=BLOCKED, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=32, name=qtp1128420885-32 Acceptor1 SelectChannelConnector@0.0.0.0:20160, state=RUNNABLE, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=31, name=qtp1128420885-31 Acceptor0 SelectChannelConnector@0.0.0.0:20160, state=BLOCKED, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=61, name=qtp1209461881-61 Acceptor2 SelectChannelConnector@0.0.0.0:32925, state=BLOCKED, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=59, name=qtp1209461881-59 Acceptor0 SelectChannelConnector@0.0.0.0:32925, state=BLOCKED, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=47, name=qtp2106422771-47 Acceptor2 SelectChannelConnector@0.0.0.0:44729, state=BLOCKED, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=46, name=qtp2106422771-46 Acceptor1 SelectChannelConnector@0.0.0.0:44729, state=BLOCKED, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=62, name=qtp1209461881-62 Acceptor3 SelectChannelConnector@0.0.0.0:32925, state=RUNNABLE, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=60, name=qtp1209461881-60 Acceptor1 SelectChannelConnector@0.0.0.0:32925, state=BLOCKED, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=34, name=qtp1128420885-34 Acceptor3 SelectChannelConnector@0.0.0.0:20160, state=BLOCKED, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=73, name=qtp1680555153-73 Acceptor1 SelectChannelConnector@0.0.0.0:46108, state=BLOCKED, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=75, name=qtp1680555153-75 Acceptor3 SelectChannelConnector@0.0.0.0:46108, state=RUNNABLE, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=45, name=qtp2106422771-45 Acceptor0 SelectChannelConnector@0.0.0.0:44729, state=BLOCKED, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=48, name=qtp2106422771-48 Acceptor3 SelectChannelConnector@0.0.0.0:44729, state=RUNNABLE, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([9F31DB9569802843]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.update.PeerSyncTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=74, name=qtp1680555153-74 Acceptor2 SelectChannelConnector@0.0.0.0:46108, state=BLOCKED, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=33, name=qtp1128420885-33 Acceptor2 SelectChannelConnector@0.0.0.0:20160, state=BLOCKED, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=31, name=qtp1128420885-31 Acceptor0 SelectChannelConnector@0.0.0.0:20160, state=BLOCKED, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=32, name=qtp1128420885-32 Acceptor1 SelectChannelConnector@0.0.0.0:20160, state=RUNNABLE, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=72, name=qtp1680555153-72 Acceptor0 SelectChannelConnector@0.0.0.0:46108, state=BLOCKED, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=46, name=qtp2106422771-46 Acceptor1 SelectChannelConnector@0.0.0.0:44729, state=BLOCKED, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=47, name=qtp2106422771-47 Acceptor2 SelectChannelConnector@0.0.0.0:44729, state=BLOCKED, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=59, name=qtp1209461881-59 Acceptor0 SelectChannelConnector@0.0.0.0:32925, state=BLOCKED, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=61, name=qtp1209461881-61 Acceptor2 SelectChannelConnector@0.0.0.0:32925, state=BLOCKED, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=62, name=qtp1209461881-62 Acceptor3 SelectChannelConnector@0.0.0.0:32925, state=RUNNABLE, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=60, name=qtp1209461881-60 Acceptor1 SelectChannelConnector@0.0.0.0:32925, state=BLOCKED, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=34, name=qtp1128420885-34 Acceptor3 SelectChannelConnector@0.0.0.0:20160, state=BLOCKED, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=73, name=qtp1680555153-73 Acceptor1 SelectChannelConnector@0.0.0.0:46108, state=BLOCKED, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=75, name=qtp1680555153-75 Acceptor3 SelectChannelConnector@0.0.0.0:46108, state=RUNNABLE, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=45, name=qtp2106422771-45 Acceptor0 SelectChannelConnector@0.0.0.0:44729, state=BLOCKED, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=48, name=qtp2106422771-48 Acceptor3 SelectChannelConnector@0.0.0.0:44729, state=RUNNABLE, group=TGRP-PeerSyncTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=74, name=qtp1680555153-74 Acceptor2 SelectChannelConnector@0.0.0.0:46108, state=BLOCKED, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=33, name=qtp1128420885-33 Acceptor2 SelectChannelConnector@0.0.0.0:20160, state=BLOCKED, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=31, name=qtp1128420885-31 Acceptor0 SelectChannelConnector@0.0.0.0:20160, state=BLOCKED, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=32, name=qtp1128420885-32 Acceptor1 SelectChannelConnector@0.0.0.0:20160, state=RUNNABLE, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=72, name=qtp1680555153-72 Acceptor0 SelectChannelConnector@0.0.0.0:46108, state=BLOCKED, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=46, name=qtp2106422771-46 Acceptor1 SelectChannelConnector@0.0.0.0:44729, state=BLOCKED, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=47, name=qtp2106422771-47 Acceptor2 SelectChannelConnector@0.0.0.0:44729, state=BLOCKED, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=59, name=qtp1209461881-59 Acceptor0 SelectChannelConnector@0.0.0.0:32925, state=BLOCKED, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=61, name=qtp1209461881-61 Acceptor2 SelectChannelConnector@0.0.0.0:32925, state=BLOCKED, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=62, name=qtp1209461881-62 Acceptor3 SelectChannelConnector@0.0.0.0:32925, state=RUNNABLE, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=60, name=qtp1209461881-60 Acceptor1 SelectChannelConnector@0.0.0.0:32925, state=BLOCKED, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=34, name=qtp1128420885-34 Acceptor3 SelectChannelConnector@0.0.0.0:20160, state=BLOCKED, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=73, name=qtp1680555153-73 Acceptor1 SelectChannelConnector@0.0.0.0:46108, state=BLOCKED, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=75, name=qtp1680555153-75 Acceptor3 SelectChannelConnector@0.0.0.0:46108, state=RUNNABLE, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=45, name=qtp2106422771-45 Acceptor0 SelectChannelConnector@0.0.0.0:44729, state=BLOCKED, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=48, name=qtp2106422771-48 Acceptor3 SelectChannelConnector@0.0.0.0:44729, state=RUNNABLE, group=TGRP-PeerSyncTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([9F31DB9569802843]:0)




Build Log:
[...truncated 8302 lines...]
[junit4:junit4] Suite: org.apache.solr.update.PeerSyncTest
[junit4:junit4]   2> 5 T24 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> 485 T24 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 794 T24 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:20160
[junit4:junit4]   2> 849 T24 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 850 T24 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 851 T24 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 904 T24 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 906 T24 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 906 T24 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 907 T24 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 908 T24 oasc.CoreContainer.<init> New CoreContainer 1169343020
[junit4:junit4]   2> 909 T24 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 910 T24 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 969 T24 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 1042 T24 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 1050 T24 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 1053 T24 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 1054 T24 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 1111 T24 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 1163 T24 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 1164 T24 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 1210 T24 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1803 T24 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1818 T24 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1821 T24 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1842 T24 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1848 T24 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1852 T24 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1855 T24 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1858 T24 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.PeerSyncTest-1351803765116/control/data/
[junit4:junit4]   2> 1858 T24 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 1860 T24 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.PeerSyncTest-1351803765116/control/data/index/
[junit4:junit4]   2> 1861 T24 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.PeerSyncTest-1351803765116/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1862 T24 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.update.PeerSyncTest-1351803765116/control/data/index forceNew:false
[junit4:junit4]   2> 1866 T24 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7a200513 lockFactory=org.apache.lucene.store.NativeFSLockFactory@373d7b4a),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1866 T24 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 1868 T24 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1868 T24 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1869 T24 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1869 T24 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1884 T24 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1891 T24 oass.SolrIndexSearcher.<init> Opening Searcher@191c9998 main
[junit4:junit4]   2> 1891 T24 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 1938 T24 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1938 T24 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1939 T24 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 1939 T24 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 1940 T24 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 1940 T24 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 1941 T24 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 1942 T24 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 1942 T24 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 1943 T24 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 1943 T24 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 1944 T24 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 1960 T39 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@191c9998 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1966 T24 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 1966 T24 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 1967 T24 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 2035 T24 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 2058 T24 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 2060 T24 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:44729
[junit4:junit4]   2> 2062 T24 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2062 T24 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 2063 T24 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2107 T24 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 2107 T24 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2108 T24 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 2108 T24 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 2109 T24 oasc.CoreContainer.<init> New CoreContainer 2130866637
[junit4:junit4]   2> 2110 T24 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2110 T24 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2141 T24 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 2157 T24 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 2158 T24 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 2160 T24 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 2161 T24 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 2202 T24 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 2253 T24 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 2254 T24 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 2293 T24 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 2944 T24 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 2961 T24 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 2965 T24 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 2989 T24 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2998 T24 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 3006 T24 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 3011 T24 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 3015 T24 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.PeerSyncTest-1351803765116/shard0/data/
[junit4:junit4]   2> 3016 T24 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 3018 T24 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.PeerSyncTest-1351803765116/shard0/data/index/
[junit4:junit4]   2> 3019 T24 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.PeerSyncTest-1351803765116/shard0/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 3021 T24 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.update.PeerSyncTest-1351803765116/shard0/data/index forceNew:false
[junit4:junit4]   2> 3027 T24 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@29e2147d lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c79f780),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 3027 T24 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 3030 T24 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 3031 T24 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 3032 T24 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 3033 T24 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 3049 T24 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 3057 T24 oass.SolrIndexSearcher.<init> Opening Searcher@5a95a3af main
[junit4:junit4]   2> 3057 T24 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3059 T24 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 3059 T24 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 3060 T24 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 3061 T24 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 3061 T24 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 3062 T24 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 3062 T24 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 3063 T24 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 3063 T24 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 3064 T24 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 3065 T24 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 3065 T24 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 3084 T53 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5a95a3af main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 3084 T24 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 3086 T24 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 3086 T24 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3087 T24 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3099 T24 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 3102 T24 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:32925
[junit4:junit4]   2> 3103 T24 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3104 T24 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 3105 T24 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 3146 T24 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 3146 T24 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3147 T24 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 3147 T24 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 3148 T24 oasc.CoreContainer.<init> New CoreContainer 1658929028
[junit4:junit4]   2> 3149 T24 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 3149 T24 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 3182 T24 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 3204 T24 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 3205 T24 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 3206 T24 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 3207 T24 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 3257 T24 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 3304 T24 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 3305 T24 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 3336 T24 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 3905 T24 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 3919 T24 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 3924 T24 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 3941 T24 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 3947 T24 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 3951 T24 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 3953 T24 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 3956 T24 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.PeerSyncTest-1351803765116/shard1/data/
[junit4:junit4]   2> 3956 T24 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 3957 T24 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.PeerSyncTest-1351803765116/shard1/data/index/
[junit4:junit4]   2> 3958 T24 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.PeerSyncTest-1351803765116/shard1/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 3959 T24 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.update.PeerSyncTest-1351803765116/shard1/data/index forceNew:false
[junit4:junit4]   2> 3963 T24 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5742691b lockFactory=org.apache.lucene.store.NativeFSLockFactory@496168c4),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 3963 T24 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 3965 T24 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 3965 T24 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 3966 T24 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 3967 T24 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 3976 T24 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 3981 T24 oass.SolrIndexSearcher.<init> Opening Searcher@1f58cce5 main
[junit4:junit4]   2> 3982 T24 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3984 T24 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 3984 T24 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 3985 T24 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 3985 T24 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 3986 T24 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 3986 T24 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 3987 T24 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 3987 T24 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 3987 T24 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 3988 T24 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 3988 T24 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 3989 T24 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 4002 T66 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1f58cce5 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 4002 T24 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 4003 T24 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 4004 T24 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 4004 T24 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 4015 T24 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 4018 T24 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:46108
[junit4:junit4]   2> 4018 T24 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4019 T24 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 4019 T24 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4049 T24 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 4050 T24 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4050 T24 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 4050 T24 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 4051 T24 oasc.CoreContainer.<init> New CoreContainer 1719281134
[junit4:junit4]   2> 4052 T24 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4052 T24 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4078 T24 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4096 T24 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 4096 T24 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 4098 T24 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 4099 T24 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4147 T24 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 4189 T24 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 4190 T24 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4215 T24 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 4758 T24 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 4771 T24 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 4775 T24 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 4792 T24 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4797 T24 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4802 T24 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4804 T24 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4806 T24 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.PeerSyncTest-1351803765116/shard2/data/
[junit4:junit4]   2> 4806 T24 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 4807 T24 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.PeerSyncTest-1351803765116/shard2/data/index/
[junit4:junit4]   2> 4808 T24 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.PeerSyncTest-1351803765116/shard2/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 4809 T24 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.update.PeerSyncTest-1351803765116/shard2/data/index forceNew:false
[junit4:junit4]   2> 4812 T24 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@57cc31bc lockFactory=org.apache.lucene.store.NativeFSLockFactory@4933e204),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 4813 T24 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 4814 T24 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 4815 T24 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 4815 T24 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 4816 T24 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 4825 T24 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 4830 T24 oass.SolrIndexSearcher.<init> Opening Searcher@22ed101f main
[junit4:junit4]   2> 4831 T24 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 4832 T24 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 4833 T24 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 4833 T24 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 4834 T24 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4834 T24 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 4835 T24 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4835 T24 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4836 T24 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4836 T24 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4837 T24 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4837 T24 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4838 T24 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 4851 T80 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@22ed101f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 4851 T24 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 4852 T24 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 4853 T24 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 4853 T24 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C1 name=collection1 org.apache.solr.core.SolrCore@54d05e46
[junit4:junit4]   2> 5842 T50 C1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@29e2147d lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c79f780),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5842 T50 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 5991 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=173 
[junit4:junit4]   2> ASYNC  NEW_CORE C2 name=collection1 org.apache.solr.core.SolrCore@295e3857
[junit4:junit4]   2> 6244 T64 C2 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 6290 T64 C2 oasu.PeerSync.sync PeerSync: core=collection1 url= START replicas=[127.0.0.1:44729/solr] nUpdates=100
[junit4:junit4]   2> 6306 T64 C2 oasu.PeerSync.sync PeerSync: core=collection1 url= DONE.  We have no versions.  sync failed.
[junit4:junit4]   2> 6307 T64 C2 REQ [collection1] webapp=/solr path=/get params={qt=/get&wt=javabin&sync=127.0.0.1:44729/solr&version=2&getVersions=100} status=0 QTime=156 
[junit4:junit4]   2> 6318 T65 C2 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5742691b lockFactory=org.apache.lucene.store.NativeFSLockFactory@496168c4),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 6320 T65 C2 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 6327 T65 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=15 
[junit4:junit4]   2> 6333 T64 C2 oasu.PeerSync.sync PeerSync: core=collection1 url= START replicas=[127.0.0.1:44729/solr] nUpdates=100
[junit4:junit4]   2> 6345 T51 C1 REQ [collection1] webapp=/solr path=/get params={distrib=false&qt=/get&wt=javabin&version=2&getVersions=100} status=0 QTime=17 
[junit4:junit4]   2> 6345 T52 C1 REQ [collection1] webapp=/solr path=/get params={distrib=false&qt=/get&wt=javabin&version=2&getVersions=100} status=0 QTime=5 
[junit4:junit4]   2> 6348 T64 C2 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=  Received 1 versions from 127.0.0.1:44729/solr
[junit4:junit4]   2> 6349 T64 C2 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=  Our versions are newer. ourLowThreshold=1 otherHigh=1
[junit4:junit4]   2> 6350 T64 C2 oasu.PeerSync.sync PeerSync: core=collection1 url= DONE. sync succeeded
[junit4:junit4]   2> 6351 T64 C2 REQ [collection1] webapp=/solr path=/get params={qt=/get&wt=javabin&sync=127.0.0.1:44729/solr&version=2&getVersions=100} status=0 QTime=18 
[junit4:junit4]   2> 6362 T50 C1 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 6703 T50 C1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@29e2147d lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c79f780),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@29e2147d lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c79f780),segFN=segments_2,generation=2,filenames=[_0.fnm, _0.pos, _0.tim, _0_nrm.cfe, segments_2, _0.fdx, _0_nrm.cfs, _0.si, _0.doc, _0.tip, _0.fdt]
[junit4:junit4]   2> 6703 T50 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 6783 T50 C1 oass.SolrIndexSearcher.<init> Opening Searcher@4324438 main
[junit4:junit4]   2> 6783 T50 C1 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 6785 T53 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4324438 main{StandardDirectoryReader(segments_2:3 _0(4.1):C1)}
[junit4:junit4]   2> 6785 T50 C1 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 6786 T50 C1 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 424
[junit4:junit4]   2> 6790 T64 C2 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 6795 T64 C2 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5742691b lockFactory=org.apache.lucene.store.NativeFSLockFactory@496168c4),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5742691b lockFactory=org.apache.lucene.store.NativeFSLockFactory@496168c4),segFN=segments_2,generation=2,filenames=[_0.fnm, _0.pos, _0.tim, _0_nrm.cfe, segments_2, _0.fdx, _0_nrm.cfs, _0.si, _0.doc, _0.tip, _0.fdt]
[junit4:junit4]   2> 6796 T64 C2 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 6799 T64 C2 oass.SolrIndexSearcher.<init> Opening Searcher@3ba472fd main
[junit4:junit4]   2> 6799 T64 C2 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 6801 T64 C2 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 6806 T66 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3ba472fd main{StandardDirectoryReader(segments_2:3 _0(4.1):C1)}
[junit4:junit4]   2> 6807 T64 C2 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 17
[junit4:junit4]   2> 6828 T51 C1 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=1 status=0 QTime=4 
[junit4:junit4]   2> 6840 T65 C2 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 6884 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=33 
[junit4:junit4]   2> 6888 T64 C2 oasu.PeerSync.sync PeerSync: core=collection1 url= START replicas=[127.0.0.1:44729/solr] nUpdates=100
[junit4:junit4]   2> 6893 T51 C1 REQ [collection1] webapp=/solr path=/get params={distrib=false&qt=/get&wt=javabin&version=2&getVersions=100} status=0 QTime=0 
[junit4:junit4]   2> 6894 T64 C2 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=  Received 2 versions from 127.0.0.1:44729/solr
[junit4:junit4]   2> 6895 T64 C2 oasu.PeerSync.requestUpdates PeerSync: core=collection1 url= Requesting updates from 127.0.0.1:44729/solrn=1 versions=[2]
[junit4:junit4]   2> 6900 T50 C1 REQ [collection1] webapp=/solr path=/get params={distrib=false&getUpdates=2&qt=/get&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 6911 T64 C2 oasu.PeerSync.sync PeerSync: core=collection1 url= DONE. sync succeeded
[junit4:junit4]   2> 6912 T64 C2 REQ [collection1] webapp=/solr path=/get params={qt=/get&wt=javabin&sync=127.0.0.1:44729/solr&version=2&getVersions=100} status=0 QTime=24 
[junit4:junit4]   2> 6917 T52 C1 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 6967 T52 C1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@29e2147d lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c79f780),segFN=segments_2,generation=2,filenames=[_0.fnm, _0.pos, _0.tim, _0_nrm.cfe, segments_2, _0.fdx, _0_nrm.cfs, _0.si, _0.doc, _0.tip, _0.fdt]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@29e2147d lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c79f780),segFN=segments_3,generation=3,filenames=[_1.doc, _1_nrm.cfs, _0.fnm, _0_nrm.cfs, _1.fnm, _1_nrm.cfe, _1.fdx, _1.fdt, _1.pos, _0.pos, _1.si, _0_nrm.cfe, _0.tim, _0.fdx, _0.si, _1.tip, _0.tip, _0.doc, _1.tim, segments_3, _0.fdt]
[junit4:junit4]   2> 6968 T52 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 3
[junit4:junit4]   2> 6976 T52 C1 oass.SolrIndexSearcher.<init> Opening Searcher@630fde26 main
[junit4:junit4]   2> 6977 T52 C1 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 6978 T53 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@630fde26 main{StandardDirectoryReader(segments_3:5 _0(4.1):C1 _1(4.1):C1)}
[junit4:junit4]   2> 6978 T52 C1 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 6978 T52 C1 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 61
[junit4:junit4]   2> 6985 T65 C2 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 7007 T65 C2 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5742691b lockFactory=org.apache.lucene.store.NativeFSLockFactory@496168c4),segFN=segments_2,generation=2,filenames=[_0.fnm, _0.pos, _0.tim, _0_nrm.cfe, segments_2, _0.fdx, _0_nrm.cfs, _0.si, _0.doc, _0.tip, _0.fdt]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5742691b lockFactory=org.apache.lucene.store.NativeFSLockFactory@496168c4),segFN=segments_3,generation=3,filenames=[_1.doc, _1_nrm.cfs, _0.fnm, _0_nrm.cfs, _1.fnm, _1_nrm.cfe, _1.fdx, _1.fdt, _1.pos, _0.pos, _1.si, _0_nrm.cfe, _0.tim, _0.fdx, _0.si, _1.tip, _0.tip, _0.doc, _1.tim, segments_3, _0.fdt]
[junit4:junit4]   2> 7008 T65 C2 oasc.SolrDeletionPolicy.updateCommits newest commit = 3
[junit4:junit4]   2> 7016 T65 C2 oass.SolrIndexSearcher.<init> Opening Searcher@24edc786 main
[junit4:junit4]   2> 7018 T65 C2 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 7020 T66 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@24edc786 main{StandardDirectoryReader(segments_3:5 _0(4.1):C1 _1(4.1):C1)}
[junit4:junit4]   2> 7020 T65 C2 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 7021 T65 C2 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 37
[junit4:junit4]   2> 7026 T51 C1 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=2 status=0 QTime=1 
[junit4:junit4]   2> 7032 T64 C2 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=2 status=0 QTime=0 
[junit4:junit4]   2> 7051 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=12 
[junit4:junit4]   2> 7060 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 7068 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 7079 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=6 
[junit4:junit4]   2> 7090 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=6 
[junit4:junit4]   2> 7098 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 7108 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=5 
[junit4:junit4]   2> 7117 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 7121 T64 C2 oasu.PeerSync.sync PeerSync: core=collection1 url= START replicas=[127.0.0.1:44729/solr] nUpdates=100
[junit4:junit4]   2> 7125 T51 C1 REQ [collection1] webapp=/solr path=/get params={distrib=false&qt=/get&wt=javabin&version=2&getVersions=100} status=0 QTime=0 
[junit4:junit4]   2> 7126 T64 C2 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=  Received 10 versions from 127.0.0.1:44729/solr
[junit4:junit4]   2> 7127 T64 C2 oasu.PeerSync.requestUpdates PeerSync: core=collection1 url= Requesting updates from 127.0.0.1:44729/solrn=8 versions=[10, 9, 8, 7, 6, 5, 4, 3]
[junit4:junit4]   2> 7132 T50 C1 REQ [collection1] webapp=/solr path=/get params={distrib=false&getUpdates=10,9,8,7,6,5,4,3&qt=/get&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7159 T64 C2 oasu.PeerSync.sync PeerSync: core=collection1 url= DONE. sync succeeded
[junit4:junit4]   2> 7160 T64 C2 REQ [collection1] webapp=/solr path=/get params={qt=/get&wt=javabin&sync=127.0.0.1:44729/solr&version=2&getVersions=100} status=0 QTime=39 
[junit4:junit4]   2> 7164 T51 C1 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 7192 T51 C1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@29e2147d lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c79f780),segFN=segments_3,generation=3,filenames=[_1.doc, _1_nrm.cfs, _0.fnm, _0_nrm.cfs, _1.fnm, _1_nrm.cfe, _1.fdx, _1.fdt, _1.pos, _0.pos, _1.si, _0_nrm.cfe, _0.tim, _0.fdx, _0.si, _1.tip, _0.tip, _0.doc, _1.tim, segments_3, _0.fdt]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@29e2147d lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c79f780),segFN=segments_4,generation=4,filenames=[_1.doc, _2.pos, _2.si, _1.fnm, _2.doc, _2.tim, _2_nrm.cfe, _1.pos, _2_nrm.cfs, _0.si, _0.doc, _1.tip, _1.tim, _1_nrm.cfs, _0.fnm, _2.fdt, _0_nrm.cfs, _2.fdx, _2.fnm, _1_nrm.cfe, _1.fdx, _2.tip, _0.pos, _1.fdt, _1.si, _0.tim, _0_nrm.cfe, _0.fdx, _0.tip, _0.fdt, segments_4]
[junit4:junit4]   2> 7193 T51 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 4
[junit4:junit4]   2> 7198 T51 C1 oass.SolrIndexSearcher.<init> Opening Searcher@44be2528 main
[junit4:junit4]   2> 7198 T51 C1 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 7199 T53 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@44be2528 main{StandardDirectoryReader(segments_4:7 _0(4.1):C1 _1(4.1):C1 _2(4.1):C8)}
[junit4:junit4]   2> 7199 T51 C1 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 7200 T51 C1 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 36
[junit4:junit4]   2> 7204 T64 C2 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 7281 T64 C2 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5742691b lockFactory=org.apache.lucene.store.NativeFSLockFactory@496168c4),segFN=segments_3,generation=3,filenames=[_1.doc, _1_nrm.cfs, _0.fnm, _0_nrm.cfs, _1.fnm, _1_nrm.cfe, _1.fdx, _1.fdt, _1.pos, _0.pos, _1.si, _0_nrm.cfe, _0.tim, _0.fdx, _0.si, _1.tip, _0.tip, _0.doc, _1.tim, segments_3, _0.fdt]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5742691b lockFactory=org.apache.lucene.store.NativeFSLockFactory@496168c4),segFN=segments_4,generation=4,filenames=[_1.doc, _2.pos, _2.si, _1.fnm, _2.doc, _2.tim, _2_nrm.cfe, _1.pos, _2_nrm.cfs, _0.si, _0.doc, _1.tip, _1.tim, _1_nrm.cfs, _0.fnm, _2.fdt, _0_nrm.cfs, _2.fdx, _2.fnm, _1_nrm.cfe, _1.fdx, _2.tip, _0.pos, _1.fdt, _1.si, _0.tim, _0_nrm.cfe, _0.fdx, _0.tip, _0.fdt, segments_4]
[junit4:junit4]   2> 7282 T64 C2 oasc.SolrDeletionPolicy.updateCommits newest commit = 4
[junit4:junit4]   2> 7288 T64 C2 oass.SolrIndexSearcher.<init> Opening Searcher@7592b321 main
[junit4:junit4]   2> 7289 T64 C2 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 7291 T66 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7592b321 main{StandardDirectoryReader(segments_4:7 _0(4.1):C1 _1(4.1):C1 _2(4.1):C8)}
[junit4:junit4]   2> 7291 T64 C2 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 7292 T64 C2 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 88
[junit4:junit4]   2> 7296 T50 C1 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=10 status=0 QTime=1 
[junit4:junit4]   2> 7305 T65 C2 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=10 status=0 QTime=0 
[junit4:junit4]   2> 7318 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 7324 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7329 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7334 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7339 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7344 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7349 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7354 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7359 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7364 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7369 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7373 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7378 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7384 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7389 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7394 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7399 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7404 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7409 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7414 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7420 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7426 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7432 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7438 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7443 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7450 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7460 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7465 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7471 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7478 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7483 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7490 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7496 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7502 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7507 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7512 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7517 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7523 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7528 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7534 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7540 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7547 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 7553 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7558 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7565 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7571 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7576 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7581 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7587 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7593 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7599 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7606 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7612 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7617 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7622 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7627 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7633 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7640 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7646 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7652 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7657 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7662 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7667 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7673 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7678 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7684 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7690 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7695 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7700 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7705 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7710 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7715 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7720 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7726 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7733 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7739 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7744 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7749 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7754 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7759 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7764 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7769 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7774 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7779 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7784 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7789 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7794 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7798 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7803 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7808 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7813 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7818 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7823 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7828 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7833 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7836 T65 C2 oasu.PeerSync.sync PeerSync: core=collection1 url= START replicas=[127.0.0.1:44729/solr] nUpdates=100
[junit4:junit4]   2> 7841 T52 C1 REQ [collection1] webapp=/solr path=/get params={distrib=false&qt=/get&wt=javabin&version=2&getVersions=100} status=0 QTime=1 
[junit4:junit4]   2> 7842 T65 C2 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=  Received 100 versions from 127.0.0.1:44729/solr
[junit4:junit4]   2> 7842 T65 C2 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=  Our versions are too old. ourHighThreshold=8 otherLowThreshold=25
[junit4:junit4]   2> 7843 T65 C2 oasu.PeerSync.sync PeerSync: core=collection1 url= DONE. sync failed
[junit4:junit4]   2> 7843 T65 C2 REQ [collection1] webapp=/solr path=/get params={qt=/get&wt=javabin&sync=127.0.0.1:44729/solr&version=2&getVersions=100} status=0 QTime=8 
[junit4:junit4]   2> 7849 T64 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 7855 T65 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7860 T64 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7865 T65 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7870 T64 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7875 T65 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7880 T64 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7885 T65 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7891 T64 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7897 T65 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7902 T64 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7908 T65 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7913 T64 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7918 T65 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7923 T64 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7928 T65 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7933 T64 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7939 T65 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7944 T64 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7949 T65 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7954 T64 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7958 T65 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7963 T64 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7967 T65 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 7970 T64 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 7973 T65 C2 oasu.PeerSync.sync PeerSync: core=collection1 url= START replicas=[127.0.0.1:44729/solr] nUpdates=100
[junit4:junit4]   2> 7978 T50 C1 REQ [collection1] webapp=/solr path=/get params={distrib=false&qt=/get&wt=javabin&version=2&getVersions=100} status=0 QTime=1 
[junit4:junit4]   2> 7979 T65 C2 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=  Received 100 versions from 127.0.0.1:44729/solr
[junit4:junit4]   2> 7980 T65 C2 oasu.PeerSync.requestUpdates PeerSync: core=collection1 url= Requesting updates from 127.0.0.1:44729/solrn=70 versions=[105, 104, 103, 102, 101, 100, 99, 98, 97, 96, 95, 94, 93, 92, 91, 90, 89, 88, 87, 86, 85, 84, 83, 82, 81, 80, 79, 78, 77, 76, 75, 74, 73, 72, 71, 70, 69, 68, 67, 66, 65, 64, 63, 62, 61, 60, 59, 58, 57, 56, 55, 54, 53, 52, 51, 50, 49, 48, 47, 46, 45, 44, 43, 42, 41, 40, 39, 38, 37, 36]
[junit4:junit4]   2> 7989 T51 C1 REQ [collection1] webapp=/solr path=/get params={distrib=false&getUpdates=105,104,103,102,101,100,99,98,97,96,95,94,93,92,91,90,89,88,87,86,85,84,83,82,81,80,79,78,77,76,75,74,73,72,71,70,69,68,67,66,65,64,63,62,61,60,59,58,57,56,55,54,53,52,51,50,49,48,47,46,45,44,43,42,41,40,39,38,37,36&qt=/get&wt=javabin&version=2} status=0 QTime=5 
[junit4:junit4]   2> 8067 T65 C2 oasu.PeerSync.sync PeerSync: core=collection1 url= DONE. sync succeeded
[junit4:junit4]   2> 8068 T65 C2 REQ [collection1] webapp=/solr path=/get params={qt=/get&wt=javabin&sync=127.0.0.1:44729/solr&version=2&getVersions=100} status=0 QTime=95 
[junit4:junit4]   2> 8073 T52 C1 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 8107 T52 C1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@29e2147d lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c79f780),segFN=segments_4,generation=4,filenames=[_1.doc, _2.pos, _2.si, _1.fnm, _2.doc, _2.tim, _2_nrm.cfe, _1.pos, _2_nrm.cfs, _0.si, _0.doc, _1.tip, _1.tim, _1_nrm.cfs, _0.fnm, _2.fdt, _0_nrm.cfs, _2.fdx, _2.fnm, _1_nrm.cfe, _1.fdx, _2.tip, _0.pos, _1.fdt, _1.si, _0.tim, _0_nrm.cfe, _0.fdx, _0.tip, _0.fdt, segments_4]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@29e2147d lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c79f780),segFN=segments_5,generation=5,filenames=[_3_nrm.cfe, _1.doc, _2.pos, _2.si, _3.tim, _3.fdt, _3.pos, _3.fnm, _1.fnm, _2.doc, _2.tim, _2_nrm.cfe, _1.pos, _2_nrm.cfs, _0.si, _0.doc, _1.tip, _1.tim, _1_nrm.cfs, _0.fnm, _3.doc, _2.fdt, _0_nrm.cfs, _2.fdx, _2.fnm, _1_nrm.cfe, _3.fdx, _3_nrm.cfs, _1.fdx, _2.tip, _0.pos, _1.fdt, _1.si, _0.tim, _0_nrm.cfe, _3.si, _3.tip, _0.fdx, segments_5, _0.tip, _0.fdt]
[junit4:junit4]   2> 8108 T52 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 5
[junit4:junit4]   2> 8112 T52 C1 oass.SolrIndexSearcher.<init> Opening Searcher@545ecbdd main
[junit4:junit4]   2> 8112 T52 C1 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 8113 T53 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@545ecbdd main{StandardDirectoryReader(segments_5:9 _0(4.1):C1 _1(4.1):C1 _2(4.1):C8 _3(4.1):C95)}
[junit4:junit4]   2> 8114 T52 C1 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 8115 T52 C1 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 42
[junit4:junit4]   2> 8119 T64 C2 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 8152 T64 C2 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5742691b lockFactory=org.apache.lucene.store.NativeFSLockFactory@496168c4),segFN=segments_4,generation=4,filenames=[_1.doc, _2.pos, _2.si, _1.fnm, _2.doc, _2.tim, _2_nrm.cfe, _1.pos, _2_nrm.cfs, _0.si, _0.doc, _1.tip, _1.tim, _1_nrm.cfs, _0.fnm, _2.fdt, _0_nrm.cfs, _2.fdx, _2.fnm, _1_nrm.cfe, _1.fdx, _2.tip, _0.pos, _1.fdt, _1.si, _0.tim, _0_nrm.cfe, _0.fdx, _0.tip, _0.fdt, segments_4]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5742691b lockFactory=org.apache.lucene.store.NativeFSLockFactory@496168c4),segFN=segments_5,generation=5,filenames=[_3_nrm.cfe, _1.doc, _2.pos, _2.si, _3.tim, _3.fdt, _3.pos, _3.fnm, _1.fnm, _2.doc, _2.tim, _2_nrm.cfe, _1.pos, _2_nrm.cfs, _0.si, _0.doc, _1.tip, _1.tim, _1_nrm.cfs, _0.fnm, _3.doc, _2.fdt, _0_nrm.cfs, _2.fdx, _2.fnm, _1_nrm.cfe, _3.fdx, _3_nrm.cfs, _1.fdx, _2.tip, _0.pos, _1.fdt, _1.si, _0.tim, _0_nrm.cfe, _3.si, _3.tip, _0.fdx, segments_5, _0.tip, _0.fdt]
[junit4:junit4]   2> 8153 T64 C2 oasc.SolrDeletionPolicy.updateCommits newest commit = 5
[junit4:junit4]   2> 8157 T64 C2 oass.SolrIndexSearcher.<init> Opening Searcher@7883822a main
[junit4:junit4]   2> 8157 T64 C2 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 8158 T66 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7883822a main{StandardDirectoryReader(segments_5:9 _0(4.1):C1 _1(4.1):C1 _2(4.1):C8 _3(4.1):C95)}
[junit4:junit4]   2> 8159 T64 C2 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 8160 T64 C2 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 41
[junit4:junit4]   2> 8312 T50 C1 REQ [collection1] webapp=/solr path=/select params={sort=_version_+desc&wt=javabin&q=*:*&version=2} hits=105 status=0 QTime=149 
[junit4:junit4]   2> 8323 T65 C2 REQ [collection1] webapp=/solr path=/select params={sort=_version_+desc&wt=javabin&q=*:*&version=2} hits=105 status=0 QTime=4 
[junit4:junit4]   2> 8333 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 8339 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 8580 T50 C1 oass.SolrIndexSearcher.<init> Opening Searcher@31ce791 realtime
[junit4:junit4]   2> 8580 T50 C1 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 8581 T50 C1 REQ [collection1] webapp=/solr path=/update params={_version_=-1003&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=238 
[junit4:junit4]   2> 8588 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 8593 T50 C1 REQ [collection1] webapp=/solr path=/update params={_version_=-1005&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 8597 T64 C2 oasu.PeerSync.sync PeerSync: core=collection1 url= START replicas=[127.0.0.1:44729/solr] nUpdates=100
[junit4:junit4]   2> 8605 T52 C1 REQ [collection1] webapp=/solr path=/get params={distrib=false&qt=/get&wt=javabin&version=2&getVersions=100} status=0 QTime=4 
[junit4:junit4]   2> 8606 T64 C2 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=  Received 100 versions from 127.0.0.1:44729/solr
[junit4:junit4]   2> 8607 T64 C2 oasu.PeerSync.requestUpdates PeerSync: core=collection1 url= Requesting updates from 127.0.0.1:44729/solrn=5 versions=[-1005, 1004, -1003, 1002, 1001]
[junit4:junit4]   2> 8612 T51 C1 REQ [collection1] webapp=/solr path=/get params={distrib=false&getUpdates=-1005,1004,-1003,1002,1001&qt=/get&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 8691 T64 C2 oass.SolrIndexSearcher.<init> Opening Searcher@6c2658b realtime
[junit4:junit4]   2> 8692 T64 C2 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 8694 T64 C2 oasu.PeerSync.sync PeerSync: core=collection1 url= DONE. sync succeeded
[junit4:junit4]   2> 8695 T64 C2 REQ [collection1] webapp=/solr path=/get params={qt=/get&wt=javabin&sync=127.0.0.1:44729/solr&version=2&getVersions=100} status=0 QTime=99 
[junit4:junit4]   2> 8699 T50 C1 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 8707 T50 C1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@29e2147d lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c79f780),segFN=segments_5,generation=5,filenames=[_3_nrm.cfe, _1.doc, _2.pos, _2.si, _3.tim, _3.fdt, _3.pos, _3.fnm, _1.fnm, _2.doc, _2.tim, _2_nrm.cfe, _1.pos, _2_nrm.cfs, _0.si, _0.doc, _1.tip, _1.tim, _1_nrm.cfs, _0.fnm, _3.doc, _2.fdt, _0_nrm.cfs, _2.fdx, _2.fnm, _1_nrm.cfe, _3.fdx, _3_nrm.cfs, _1.fdx, _2.tip, _0.pos, _1.fdt, _1.si, _0.tim, _0_nrm.cfe, _3.si, _3.tip, _0.fdx, segments_5, _0.tip, _0.fdt]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@29e2147d lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c79f780),segFN=segments_6,generation=6,filenames=[_5_nrm.cfe, _3_nrm.cfe, _1.doc, _2.pos, _2.si, _5.fdt, _3.tim, _3.fdt, _3.pos, _3.fnm, _1.fnm, _2.doc, _2.tim, _5.tip, _5.tim, _2_nrm.cfe, _1.pos, _2_nrm.cfs, _0.si, _0.doc, _1.tip, _1.tim, _1_nrm.cfs, _5.fnm, _0.fnm, _5.pos, _3.doc, _2.fdt, _5.doc, _0_nrm.cfs, _2.fdx, _2.fnm, _1_nrm.cfe, _5.si, _3.fdx, _3_nrm.cfs, _1.fdx, _2.tip, _5_nrm.cfs, _0.pos, _1.fdt, _1.si, _0.tim, _0_nrm.cfe, _5.fdx, _3.si, _3.tip, _0.fdx, _0.tip, segments_6, _0.fdt]
[junit4:junit4]   2> 8707 T50 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 6
[junit4:junit4]   2> 8709 T50 C1 oass.SolrIndexSearcher.<init> Opening Searcher@1f81719d main
[junit4:junit4]   2> 8710 T50 C1 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 8711 T53 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1f81719d main{StandardDirectoryReader(segments_6:16:nrt _0(4.1):C1 _1(4.1):C1 _2(4.1):C8 _3(4.1):C95 _5(4.1):C1)}
[junit4:junit4]   2> 8711 T50 C1 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 8713 T50 C1 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 14
[junit4:junit4]   2> 8717 T65 C2 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 8723 T65 C2 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5742691b lockFactory=org.apache.lucene.store.NativeFSLockFactory@496168c4),segFN=segments_5,generation=5,filenames=[_3_nrm.cfe, _1.doc, _2.pos, _2.si, _3.tim, _3.fdt, _3.pos, _3.fnm, _1.fnm, _2.doc, _2.tim, _2_nrm.cfe, _1.pos, _2_nrm.cfs, _0.si, _0.doc, _1.tip, _1.tim, _1_nrm.cfs, _0.fnm, _3.doc, _2.fdt, _0_nrm.cfs, _2.fdx, _2.fnm, _1_nrm.cfe, _3.fdx, _3_nrm.cfs, _1.fdx, _2.tip, _0.pos, _1.fdt, _1.si, _0.tim, _0_nrm.cfe, _3.si, _3.tip, _0.fdx, segments_5, _0.tip, _0.fdt]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5742691b lockFactory=org.apache.lucene.store.NativeFSLockFactory@496168c4),segFN=segments_6,generation=6,filenames=[_5_nrm.cfe, _3_nrm.cfe, _1.doc, _2.pos, _2.si, _5.fdt, _3.tim, _3.fdt, _3.pos, _3.fnm, _1.fnm, _2.doc, _2.tim, _5.tip, _5.tim, _2_nrm.cfe, _1.pos, _2_nrm.cfs, _0.si, _0.doc, _1.tip, _1.tim, _1_nrm.cfs, _5.fnm, _0.fnm, _5.pos, _3.doc, _2.fdt, _5.doc, _0_nrm.cfs, _2.fdx, _2.fnm, _1_nrm.cfe, _5.si, _3.fdx, _3_nrm.cfs, _1.fdx, _2.tip, _5_nrm.cfs, _0.pos, _1.fdt, _1.si, _0.tim, _0_nrm.cfe, _5.fdx, _3.si, _3.tip, _0.fdx, _0.tip, segments_6, _0.fdt]
[junit4:junit4]   2> 8724 T65 C2 oasc.SolrDeletionPolicy.updateCommits newest commit = 6
[junit4:junit4]   2> 8727 T65 C2 oass.SolrIndexSearcher.<init> Opening Searcher@1b02293 main
[junit4:junit4]   2> 8727 T65 C2 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 8729 T66 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b02293 main{StandardDirectoryReader(segments_6:16:nrt _0(4.1):C1 _1(4.1):C1 _2(4.1):C8 _3(4.1):C95 _5(4.1):C1)}
[junit4:junit4]   2> 8730 T65 C2 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 8730 T65 C2 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 13
[junit4:junit4]   2> 8737 T52 C1 REQ [collection1] webapp=/solr path=/select params={sort=_version_+desc&wt=javabin&q=*:*&version=2} hits=106 status=0 QTime=4 
[junit4:junit4]   2> 8744 T64 C2 REQ [collection1] webapp=/solr path=/select params={sort=_version_+desc&wt=javabin&q=*:*&version=2} hits=106 status=0 QTime=2 
[junit4:junit4]   2> 8753 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 8758 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 8769 T50 C1 oass.SolrIndexSearcher.<init> Opening Searcher@41016a75 realtime
[junit4:junit4]   2> 8770 T50 C1 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 8770 T50 C1 REQ [collection1] webapp=/solr path=/update params={_version_=-2003&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=9 
[junit4:junit4]   2> 8776 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 8780 T52 C1 REQ [collection1] webapp=/solr path=/update params={_version_=-2005&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 8785 T65 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 8795 T64 C2 oass.SolrIndexSearcher.<init> Opening Searcher@a3181c9 realtime
[junit4:junit4]   2> 8796 T64 C2 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 8797 T64 C2 REQ [collection1] webapp=/solr path=/update params={_version_=-2003&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=8 
[junit4:junit4]   2> 8802 T65 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 8806 T64 C2 REQ [collection1] webapp=/solr path=/update params={_version_=-2005&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 8809 T65 C2 oasu.PeerSync.sync PeerSync: core=collection1 url= START replicas=[127.0.0.1:44729/solr] nUpdates=100
[junit4:junit4]   2> 8813 T50 C1 REQ [collection1] webapp=/solr path=/get params={distrib=false&qt=/get&wt=javabin&version=2&getVersions=100} status=0 QTime=0 
[junit4:junit4]   2> 8814 T65 C2 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=  Received 100 versions from 127.0.0.1:44729/solr
[junit4:junit4]   2> 8815 T65 C2 oasu.PeerSync.requestUpdates PeerSync: core=collection1 url= Requesting updates from 127.0.0.1:44729/solrn=1 versions=[2002]
[junit4:junit4]   2> 8819 T52 C1 REQ [collection1] webapp=/solr path=/get params={distrib=false&getUpdates=2002&qt=/get&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 8821 T65 C2 oasu.DirectUpdateHandler2.addDoc Reordered DBQs detected.  Update=add{flags=12,_version_=2002,id=2001} DBQs=[DBQ{version=2003,q=id:2001 OR id:2002}]
[junit4:junit4]   2> 8831 T65 C2 oass.SolrIndexSearcher.<init> Opening Searcher@3cd92923 realtime
[junit4:junit4]   2> 8832 T65 C2 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 8839 T65 C2 oasu.PeerSync.sync PeerSync: core=collection1 url= DONE. sync succeeded
[junit4:junit4]   2> 8840 T65 C2 REQ [collection1] webapp=/solr path=/get params={qt=/get&wt=javabin&sync=127.0.0.1:44729/solr&version=2&getVersions=100} status=0 QTime=31 
[junit4:junit4]   2> 8844 T51 C1 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 8850 T51 C1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@29e2147d lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c79f780),segFN=segments_6,generation=6,filenames=[_5_nrm.cfe, _3_nrm.cfe, _1.doc, _2.pos, _2.si, _5.fdt, _3.tim, _3.fdt, _3.pos, _3.fnm, _1.fnm, _2.doc, _2.tim, _5.tip, _5.tim, _2_nrm.cfe, _1.pos, _2_nrm.cfs, _0.si, _0.doc, _1.tip, _1.tim, _1_nrm.cfs, _5.fnm, _0.fnm, _5.pos, _3.doc, _2.fdt, _5.doc, _0_nrm.cfs, _2.fdx, _2.fnm, _1_nrm.cfe, _5.si, _3.fdx, _3_nrm.cfs, _1.fdx, _2.tip, _5_nrm.cfs, _0.pos, _1.fdt, _1.si, _0.tim, _0_nrm.cfe, _5.fdx, _3.si, _3.tip, _0.fdx, _0.tip, segments_6, _0.fdt]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@29e2147d lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c79f780),segFN=segments_7,generation=7,filenames=[_1.doc, _3_nrm.cfe, _5_nrm.cfe, _5.fdt, _7.si, _3.fdt, _1.fnm, _2.doc, _2.tim, _7.fnm, _5.tip, _5.tim, _7.tim, _7.tip, _0.fnm, _2.fdt, _3.doc, _7.fdx, _5.doc, _2.fdx, _7.fdt, _5.si, _2.tip, _1.fdx, _3_nrm.cfs, _1.fdt, _5_nrm.cfs, _0.tim, _3.si, _0.tip, _7.doc, _2.si, _2.pos, _3.tim, _3.fnm, _3.pos, _2_nrm.cfe, _7_nrm.cfs, _1.pos, _0.si, _2_nrm.cfs, _1.tip, _0.doc, _1.tim, _1_nrm.cfs, _5.fnm, _7_nrm.cfe, _7.pos, _5.pos, _0_nrm.cfs, _1_nrm.cfe, _2.fnm, _3.fdx, segments_7, _0.pos, _1.si, _0_nrm.cfe, _5.fdx, _0.fdx, _3.tip, _0.fdt]
[junit4:junit4]   2> 8851 T51 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 7
[junit4:junit4]   2> 8853 T51 C1 oass.SolrIndexSearcher.<init> Opening Searcher@49e3c29e main
[junit4:junit4]   2> 8853 T51 C1 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 8855 T53 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@49e3c29e main{StandardDirectoryReader(segments_7:23:nrt _0(4.1):C1 _1(4.1):C1 _2(4.1):C8 _3(4.1):C95 _5(4.1):C1 _7(4.1):C1)}
[junit4:junit4]   2> 8855 T51 C1 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 8855 T51 C1 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 11
[junit4:junit4]   2> 8859 T64 C2 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 8861 T64 C2 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5742691b lockFactory=org.apache.lucene.store.NativeFSLockFactory@496168c4),segFN=segments_6,generation=6,filenames=[_5_nrm.cfe, _3_nrm.cfe, _1.doc, _2.pos, _2.si, _5.fdt, _3.tim, _3.fdt, _3.pos, _3.fnm, _1.fnm, _2.doc, _2.tim, _5.tip, _5.tim, _2_nrm.cfe, _1.pos, _2_nrm.cfs, _0.si, _0.doc, _1.tip, _1.tim, _1_nrm.cfs, _5.fnm, _0.fnm, _5.pos, _3.doc, _2.fdt, _5.doc, _0_nrm.cfs, _2.fdx, _2.fnm, _1_nrm.cfe, _5.si, _3.fdx, _3_nrm.cfs, _1.fdx, _2.tip, _5_nrm.cfs, _0.pos, _1.fdt, _1.si, _0.tim, _0_nrm.cfe, _5.fdx, _3.si, _3.tip, _0.fdx, _0.tip, segments_6, _0.fdt]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5742691b lockFactory=org.apache.lucene.store.NativeFSLockFactory@496168c4),segFN=segments_7,generation=7,filenames=[_7_1.del, _1.doc, _3_nrm.cfe, _5_nrm.cfe, _5.fdt, _7.si, _3.fdt, _1.fnm, _2.doc, _2.tim, _7.fnm, _5.tip, _5.tim, _7.tim, _7.tip, _0.fnm, _2.fdt, _3.doc, _7.fdx, _5.doc, _2.fdx, _7.fdt, _5.si, _2.tip, _1.fdx, _3_nrm.cfs, _1.fdt, _5_nrm.cfs, _0.tim, _3.si, _0.tip, _7.doc, _2.si, _2.pos, _3.tim, _3.fnm, _3.pos, _2_nrm.cfe, _7_nrm.cfs, _1.pos, _0.si, _2_nrm.cfs, _1.tip, _0.doc, _1.tim, _1_nrm.cfs, _5.fnm, _7_nrm.cfe, _7.pos, _5.pos, _0_nrm.cfs, _1_nrm.cfe, _2.fnm, _3.fdx, segments_7, _0.pos, _1.si, _0_nrm.cfe, _5.fdx, _0.fdx, _3.tip, _0.fdt]
[junit4:junit4]   2> 8862 T64 C2 oasc.SolrDeletionPolicy.updateCommits newest commit = 7
[junit4:junit4]   2> 8863 T64 C2 oass.SolrIndexSearcher.<init> Opening Searcher@32afbbca main
[junit4:junit4]   2> 8863 T64 C2 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 8864 T66 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@32afbbca main{StandardDirectoryReader(segments_6:22:nrt _0(4.1):C1 _1(4.1):C1 _2(4.1):C8 _3(4.1):C95 _5(4.1):C1 _7(4.1):C2/1)}
[junit4:junit4]   2> 8864 T64 C2 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 8865 T64 C2 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 6
[junit4:junit4]   2> 8868 T50 C1 REQ [collection1] webapp=/solr path=/select params={sort=_version_+desc&wt=javabin&q=*:*&version=2} hits=107 status=0 QTime=1 
[junit4:junit4]   2> 8874 T65 C2 REQ [collection1] webapp=/solr path=/select params={sort=_version_+desc&wt=javabin&q=*:*&version=2} hits=107 status=0 QTime=1 
[junit4:junit4]   2> 8883 T51 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 8898 T64 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 8902 T50 C1 REQ [collection1] webapp=/solr path=/update params={_version_=3000&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 8907 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 8913 T51 C1 REQ [collection1] webapp=/solr path=/update params={_version_=3001&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 8917 T65 C2 REQ [collection1] webapp=/solr path=/update params={_version_=3001&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 8922 T50 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 8927 T52 C1 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 8933 T64 C2 REQ [collection1] webapp=/solr path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 8937 T65 C2 oasu.PeerSync.sync PeerSync: core=collection1 url= START replicas=[127.0.0.1:44729/solr] nUpdates=100
[junit4:junit4]   2> 8942 T51 C1 REQ [collection1] webapp=/solr path=/get params={distrib=false&qt=/get&wt=javabin&version=2&getVersions=100} status=0 QTime=1 
[junit4:junit4]   2> 8943 T65 C2 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=  Received 100 versions from 127.0.0.1:44729/solr
[junit4:junit4]   2> 8943 T65 C2 oasu.PeerSync.requestUpdates PeerSync: core=collection1 url= Requesting updates from 127.0.0.1:44729/solrn=2 versions=[3004, 3003]
[junit4:junit4]   2> 8948 T50 C1 REQ [collection1] webapp=/solr path=/get params={distrib=false&getUpdates=3004,3003&qt=/get&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 8950 T65 C2 oasu.PeerSync.sync PeerSync: core=collection1 url= DONE. sync succeeded
[junit4:junit4]   2> 8951 T65 C2 REQ [collection1] webapp=/solr path=/get params={qt=/get&wt=javabin&sync=127.0.0.1:44729/solr&version=2&getVersions=100} status=0 QTime=15 
[junit4:junit4]   2> 8955 T52 C1 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 8961 T52 C1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@29e2147d lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c79f780),segFN=segments_7,generation=7,filenames=[_1.doc, _3_nrm.cfe, _5_nrm.cfe, _5.fdt, _7.si, _3.fdt, _1.fnm, _2.doc, _2.tim, _7.fnm, _5.tip, _5.tim, _7.tim, _7.tip, _0.fnm, _2.fdt, _3.doc, _7.fdx, _5.doc, _2.fdx, _7.fdt, _5.si, _2.tip, _1.fdx, _3_nrm.cfs, _1.fdt, _5_nrm.cfs, _0.tim, _3.si, _0.tip, _7.doc, _2.si, _2.pos, _3.tim, _3.fnm, _3.pos, _2_nrm.cfe, _7_nrm.cfs, _1.pos, _0.si, _2_nrm.cfs, _1.tip, _0.doc, _1.tim, _1_nrm.cfs, _5.fnm, _7_nrm.cfe, _7.pos, _5.pos, _0_nrm.cfs, _1_nrm.cfe, _2.fnm, _3.fdx, segments_7, _0.pos, _1.si, _0_nrm.cfe, _5.fdx, _0.fdx, _3.tip, _0.fdt]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@29e2147d lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c79f780),segFN=segments_8,generation=8,filenames=[_1.doc, _3_nrm.cfe, _5_nrm.cfe, _8_nrm.cfe, _5.fdt, _7.si, _8.tim, _3.fdt, _1.fnm, _8.tip, _8_1.del, _2.doc, _2.tim, _8.si, _7.fnm, _5.tip, _5.tim, _7.tim, _7.tip, _0.fnm, _2.fdt, _3.doc, _7.fdx, _5.doc, _2.fdx, _7.fdt, _8_nrm.cfs, _5.si, _2.tip, _1.fdx, _3_nrm.cfs, _8.fnm, _1.fdt, _5_nrm.cfs, _0.tim, _3.si, _0.tip, _7.doc, _2.si, _2.pos, _8.doc, _3.tim, _3.fnm, _3.pos, _2_nrm.cfe, _7_nrm.cfs, _1.pos, _0.si, _2_nrm.cfs, _1.tip, _0.doc, _1.tim, _1_nrm.cfs, _5.fnm, _7_nrm.cfe, _7.pos, _5.pos, _0_nrm.cfs, _1_nrm.cfe, _2.fnm, _3.fdx, segments_8, _8.pos, _0.pos, _1.si, _0_nrm.cfe, _5.fdx, _0.fdx, _3.tip, _8.fdx, _0.fdt, _8.fdt]
[junit4:junit4]   2> 8961 T52 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 8
[junit4:junit4]   2> 8963 T52 C1 oass.SolrIndexSearcher.<init> Opening Searcher@24e4ec92 main
[junit4:junit4]   2> 8964 T52 C1 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 8965 T52 C1 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 8965 T53 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@24e4ec92 main{StandardDirectoryReader(segments_8:25:nrt _0(4.1):C1 _1(4.1):C1 _2(4.1):C8 _3(4.1):C95 _5(4.1):C1 _7(4.1):C1 _8(4.1):C4/1)}
[junit4:junit4]   2> 8966 T52 C1 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 11
[junit4:junit4]   2> 8970 T64 C2 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 8975 T64 C2 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5742691b lockFactory=org.apache.lucene.store.NativeFSLockFactory@496168c4),segFN=segments_7,generation=7,filenames=[_7_1.del, _1.doc, _3_nrm.cfe, _5_nrm.cfe, _5.fdt, _7.si, _3.fdt, _1.fnm, _2.doc, _2.tim, _7.fnm, _5.tip, _5.tim, _7.tim, _7.tip, _0.fnm, _2.fdt, _3.doc, _7.fdx, _5.doc, _2.fdx, _7.fdt, _5.si, _2.tip, _1.fdx, _3_nrm.cfs, _1.fdt, _5_nrm.cfs, _0.tim, _3.si, _0.tip, _7.doc, _2.si, _2.pos, _3.tim, _3.fnm, _3.pos, _2_nrm.cfe, _7_nrm.cfs, _1.pos, _0.si, _2_nrm.cfs, _1.tip, _0.doc, _1.tim, _1_nrm.cfs, _5.fnm, _7_nrm.cfe, _7.pos, _5.pos, _0_nrm.cfs, _1_nrm.cfe, _2.fnm, _3.fdx, segments_7, _0.pos, _1.si, _0_nrm.cfe, _5.fdx, _0.fdx, _3.tip, _0.fdt]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5742691b lockFactory=org.apache.lucene.store.NativeFSLockFactory@496168c4),segFN=segments_8,generation=8,filenames=[_7_1.del, _1.doc, _3_nrm.cfe, _5_nrm.cfe, _8_nrm.cfe, _5.fdt, _7.si, _8.tim, _3.fdt, _1.fnm, _8.tip, _2.doc, _2.tim, _8.si, _7.fnm, _5.tip, _5.tim, _7.tim, _7.tip, _0.fnm, _2.fdt, _3.doc, _7.fdx, _5.doc, _2.fdx, _7.fdt, _8_nrm.cfs, _5.si, _2.tip, _1.fdx, _3_nrm.cfs, _8.fnm, _1.fdt, _5_nrm.cfs, _0.tim, _3.si, _0.tip, _7.doc, _2.si, _2.pos, _8.doc, _3.tim, _3.fnm, _3.pos, _2_nrm.cfe, _7_nrm.cfs, _1.pos, _0.si, _2_nrm.cfs, _1.tip, _0.doc, _1.tim, _1_nrm.cfs, _5.fnm, _7_nrm.cfe, _7.pos, _5.pos, _0_nrm.cfs, _1_nrm.cfe, _2.fnm, _3.fdx, segments_8, _8.pos, _0.pos, _1.si, _0_nrm.cfe, _5.fdx, _0.fdx, _3.tip, _8.fdx, _0.fdt, _8.fdt]
[junit4:junit4]   2> 8976 T64 C2 oasc.SolrDeletionPolicy.updateCommits newest commit = 8
[junit4:junit4]   2> 8978 T64 C2 oass.SolrIndexSearcher.<init> Opening Searcher@39731fbd main
[junit4:junit4]   2> 8978 T64 C2 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 8979 T66 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@39731fbd main{StandardDirectoryReader(segments_8:24:nrt _0(4.1):C1 _1(4.1):C1 _2(4.1):C8 _3(4.1):C95 _5(4.1):C1 _7(4.1):C2/1 _8(4.1):C3)}
[junit4:junit4]   2> 8979 T64 C2 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 8979 T64 C2 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 9
[junit4:junit4]   2> 8985 T51 C1 REQ [collection1] webapp=/solr path=/select params={sort=_version_+desc&wt=javabin&q=*:*&version=2} hits=110 status=0 QTime=2 
[junit4:junit4]   2> 8991 T65 C2 REQ [collection1] webapp=/solr path=/select params={sort=_version_+desc&wt=javabin&q=*:*&version=2} hits=110 status=0 QTime=1 
[junit4:junit4]   2> 9040 T24 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1169343020
[junit4:junit4]   2> 9041 T24 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@1cb37fdb
[junit4:junit4]   2> 9041 T24 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 9042 T24 oasc.SolrCore.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 9042 T24 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 9042 T24 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 9043 T24 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 9044 T24 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 9145 T24 oejut.QueuedThreadPool.doStop WARNING 4 threads could not be stopped
[junit4:junit4]   2> 9179 T24 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=2130866637
[junit4:junit4]   2> 9180 T24 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@54d05e46
[junit4:junit4]   2> 9181 T24 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=7,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=115,cumulative_deletesById=3,cumulative_deletesByQuery=2,cumulative_errors=0}
[junit4:junit4]   2> 9181 T24 oasc.SolrCore.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 9181 T24 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 9182 T24 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 9184 T24 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 9396 T24 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 9498 T24 oejut.QueuedThreadPool.doStop WARNING 4 threads could not be stopped
[junit4:junit4]   2> 9521 T24 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1658929028
[junit4:junit4]   2> 9521 T24 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@295e3857
[junit4:junit4]   2> 9522 T24 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=7,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=114,cumulative_deletesById=3,cumulative_deletesByQuery=3,cumulative_errors=0}
[junit4:junit4]   2> 9522 T24 oasc.SolrCore.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 9523 T24 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 9523 T24 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 9525 T24 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 9581 T24 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 9683 T24 oejut.QueuedThreadPool.doStop WARNING 4 threads could not be stopped
[junit4:junit4]   2> 9727 T24 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1719281134
[junit4:junit4]   2> 9728 T24 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@130f7b39
[junit4:junit4]   2> 9728 T24 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 9729 T24 oasc.SolrCore.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 9729 T24 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 9730 T24 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 9730 T24 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 9731 T24 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 9832 T24 oejut.QueuedThreadPool.doStop WARNING 4 threads could not be stopped
[junit4:junit4]   2> 9845 T24 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4]   2> 9848 T24 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> 9854 T23 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 16 leaked thread(s).
[junit4:junit4]   2> 30114 T23 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 16 threads leaked from SUITE scope at org.apache.solr.update.PeerSyncTest: 
[junit4:junit4]   2> 	   1) Thread[id=74, name=qtp1680555153-74 Acceptor2 SelectChannelConnector@0.0.0.0:46108, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   2) Thread[id=33, name=qtp1128420885-33 Acceptor2 SelectChannelConnector@0.0.0.0:20160, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   3) Thread[id=72, name=qtp1680555153-72 Acceptor0 SelectChannelConnector@0.0.0.0:46108, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   4) Thread[id=32, name=qtp1128420885-32 Acceptor1 SelectChannelConnector@0.0.0.0:20160, state=RUNNABLE, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   5) Thread[id=31, name=qtp1128420885-31 Acceptor0 SelectChannelConnector@0.0.0.0:20160, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   6) Thread[id=61, name=qtp1209461881-61 Acceptor2 SelectChannelConnector@0.0.0.0:32925, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   7) Thread[id=59, name=qtp1209461881-59 Acceptor0 SelectChannelConnector@0.0.0.0:32925, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   8) Thread[id=47, name=qtp2106422771-47 Acceptor2 SelectChannelConnector@0.0.0.0:44729, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   9) Thread[id=46, name=qtp2106422771-46 Acceptor1 SelectChannelConnector@0.0.0.0:44729, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  10) Thread[id=62, name=qtp1209461881-62 Acceptor3 SelectChannelConnector@0.0.0.0:32925, state=RUNNABLE, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  11) Thread[id=60, name=qtp1209461881-60 Acceptor1 SelectChannelConnector@0.0.0.0:32925, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  12) Thread[id=34, name=qtp1128420885-34 Acceptor3 SelectChannelConnector@0.0.0.0:20160, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  13) Thread[id=73, name=qtp1680555153-73 Acceptor1 SelectChannelConnector@0.0.0.0:46108, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  14) Thread[id=75, name=qtp1680555153-75 Acceptor3 SelectChannelConnector@0.0.0.0:46108, state=RUNNABLE, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  15) Thread[id=45, name=qtp2106422771-45 Acceptor0 SelectChannelConnector@0.0.0.0:44729, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  16) Thread[id=48, name=qtp2106422771-48 Acceptor3 SelectChannelConnector@0.0.0.0:44729, state=RUNNABLE, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 30118 T23 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
[junit4:junit4]   2> 	   1) Thread[id=74, name=qtp1680555153-74 Acceptor2 SelectChannelConnector@0.0.0.0:46108, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	   2) Thread[id=33, name=qtp1128420885-33 Acceptor2 SelectChannelConnector@0.0.0.0:20160, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	   3) Thread[id=72, name=qtp1680555153-72 Acceptor0 SelectChannelConnector@0.0.0.0:46108, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	   4) Thread[id=32, name=qtp1128420885-32 Acceptor1 SelectChannelConnector@0.0.0.0:20160, state=RUNNABLE, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	   5) Thread[id=31, name=qtp1128420885-31 Acceptor0 SelectChannelConnector@0.0.0.0:20160, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	   6) Thread[id=61, name=qtp1209461881-61 Acceptor2 SelectChannelConnector@0.0.0.0:32925, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	   7) Thread[id=59, name=qtp1209461881-59 Acceptor0 SelectChannelConnector@0.0.0.0:32925, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	   8) Thread[id=47, name=qtp2106422771-47 Acceptor2 SelectChannelConnector@0.0.0.0:44729, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	   9) Thread[id=46, name=qtp2106422771-46 Acceptor1 SelectChannelConnector@0.0.0.0:44729, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	  10) Thread[id=62, name=qtp1209461881-62 Acceptor3 SelectChannelConnector@0.0.0.0:32925, state=RUNNABLE, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	  11) Thread[id=60, name=qtp1209461881-60 Acceptor1 SelectChannelConnector@0.0.0.0:32925, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	  12) Thread[id=34, name=qtp1128420885-34 Acceptor3 SelectChannelConnector@0.0.0.0:20160, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	  13) Thread[id=73, name=qtp1680555153-73 Acceptor1 SelectChannelConnector@0.0.0.0:46108, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	  14) Thread[id=75, name=qtp1680555153-75 Acceptor3 SelectChannelConnector@0.0.0.0:46108, state=RUNNABLE, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	  15) Thread[id=45, name=qtp2106422771-45 Acceptor0 SelectChannelConnector@0.0.0.0:44729, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	  16) Thread[id=48, name=qtp2106422771-48 Acceptor3 SelectChannelConnector@0.0.0.0:44729, state=RUNNABLE, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 33308 T23 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
[junit4:junit4]   2> 	   1) Thread[id=74, name=qtp1680555153-74 Acceptor2 SelectChannelConnector@0.0.0.0:46108, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   2) Thread[id=33, name=qtp1128420885-33 Acceptor2 SelectChannelConnector@0.0.0.0:20160, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   3) Thread[id=31, name=qtp1128420885-31 Acceptor0 SelectChannelConnector@0.0.0.0:20160, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   4) Thread[id=32, name=qtp1128420885-32 Acceptor1 SelectChannelConnector@0.0.0.0:20160, state=RUNNABLE, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   5) Thread[id=72, name=qtp1680555153-72 Acceptor0 SelectChannelConnector@0.0.0.0:46108, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   6) Thread[id=46, name=qtp2106422771-46 Acceptor1 SelectChannelConnector@0.0.0.0:44729, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   7) Thread[id=47, name=qtp2106422771-47 Acceptor2 SelectChannelConnector@0.0.0.0:44729, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   8) Thread[id=59, name=qtp1209461881-59 Acceptor0 SelectChannelConnector@0.0.0.0:32925, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   9) Thread[id=61, name=qtp1209461881-61 Acceptor2 SelectChannelConnector@0.0.0.0:32925, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  10) Thread[id=62, name=qtp1209461881-62 Acceptor3 SelectChannelConnector@0.0.0.0:32925, state=RUNNABLE, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  11) Thread[id=60, name=qtp1209461881-60 Acceptor1 SelectChannelConnector@0.0.0.0:32925, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  12) Thread[id=34, name=qtp1128420885-34 Acceptor3 SelectChannelConnector@0.0.0.0:20160, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  13) Thread[id=73, name=qtp1680555153-73 Acceptor1 SelectChannelConnector@0.0.0.0:46108, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  14) Thread[id=75, name=qtp1680555153-75 Acceptor3 SelectChannelConnector@0.0.0.0:46108, state=RUNNABLE, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  15) Thread[id=45, name=qtp2106422771-45 Acceptor0 SelectChannelConnector@0.0.0.0:44729, state=BLOCKED, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  16) Thread[id=48, name=qtp2106422771-48 Acceptor3 SelectChannelConnector@0.0.0.0:44729, state=RUNNABLE, group=TGRP-PeerSyncTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Asserting, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=sr, timezone=America/Jujuy
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=17,free=90568424,total=137363456
[junit4:junit4]   2> NOTE: All tests run in this JVM: [XmlUpdateRequestHandlerTest, RAMDirectoryFa

[...truncated too long message...]

true,coord=yes): {}, locale=es_DO, timezone=Canada/East-Saskatchewan
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=5,free=74111112,total=137363456
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestIndexingPerformance, SampleTest, SimplePostToolTest, TestSearchPerf, TestCoreContainer, TestFiltering, TestLuceneMatchVersion, TestQuerySenderNoQuery, TestSuggestSpellingConverter, TestPHPSerializedResponseWriter, TestXIncludeConfig, TestRTGBase, ScriptEngineTest, JsonLoaderTest, WordBreakSolrSpellCheckerTest, TestPerFieldSimilarity, CSVRequestHandlerTest, TestBadConfig, SOLR749Test, DateFieldTest, CopyFieldTest, SearchHandlerTest, TestJoin, SortByFunctionTest, MoreLikeThisHandlerTest, TestFastLRUCache, TestJmxMonitoredMap, JSONWriterTest, MinimalSchemaTest, TestSolrIndexConfig, SystemInfoHandlerTest, TestIBSimilarityFactory, PathHierarchyTokenizerFactoryTest, SpellPossibilityIteratorTest, SimpleFacetsTest, SolrCoreCheckLockOnStartupTest, TestBinaryResponseWriter, TestReload, TestLFUCache, TestPseudoReturnFields, SuggesterFSTTest, TestSolrQueryParser, PluginInfoTest, DefaultValueUpdateProcessorTest, PreAnalyzedFieldTest, BinaryUpdateRequestHandlerTest, TestFastWriter, TestExtendedDismaxParser, DirectSolrConnectionTest, TestLMJelinekMercerSimilarityFactory, FastVectorHighlighterTest, FileBasedSpellCheckerTest, ZkControllerTest, CacheHeaderTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CacheHeaderTest -Dtests.seed=9F31DB9569802843 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_DO -Dtests.timezone=Canada/East-Saskatchewan -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR   0.00s J0 | CacheHeaderTest (suite) <<<
[junit4:junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.servlet.CacheHeaderTest: 
[junit4:junit4]    >    1) Thread[id=321, name=qtp77073360-321 Acceptor3 SelectChannelConnector@0.0.0.0:23627, state=BLOCKED, group=TGRP-CacheHeaderTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=318, name=qtp77073360-318 Acceptor0 SelectChannelConnector@0.0.0.0:23627, state=BLOCKED, group=TGRP-CacheHeaderTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=319, name=qtp77073360-319 Acceptor1 SelectChannelConnector@0.0.0.0:23627, state=BLOCKED, group=TGRP-CacheHeaderTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=320, name=qtp77073360-320 Acceptor2 SelectChannelConnector@0.0.0.0:23627, state=RUNNABLE, group=TGRP-CacheHeaderTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([9F31DB9569802843]:0)
[junit4:junit4]    > Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4]    >    1) Thread[id=321, name=qtp77073360-321 Acceptor3 SelectChannelConnector@0.0.0.0:23627, state=BLOCKED, group=TGRP-CacheHeaderTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=318, name=qtp77073360-318 Acceptor0 SelectChannelConnector@0.0.0.0:23627, state=BLOCKED, group=TGRP-CacheHeaderTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=319, name=qtp77073360-319 Acceptor1 SelectChannelConnector@0.0.0.0:23627, state=BLOCKED, group=TGRP-CacheHeaderTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=320, name=qtp77073360-320 Acceptor2 SelectChannelConnector@0.0.0.0:23627, state=RUNNABLE, group=TGRP-CacheHeaderTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([9F31DB9569802843]:0)
[junit4:junit4] Completed on J0 in 26.52s, 5 tests, 2 errors <<< FAILURES!

[...truncated 20 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:335: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/module-build.xml:61: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:1163: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:827: There were test failures: 233 suites, 965 tests, 4 suite-level errors, 675 ignored (2 assumptions)

Total time: 37 minutes 8 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure

[JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 604 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.

Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/604/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.TestSolrCoreProperties

Error Message:
4 threads leaked from SUITE scope at org.apache.solr.TestSolrCoreProperties:     1) Thread[id=40, name=qtp273092679-40 Acceptor1 SelectChannelConnector@0.0.0.0:63924, state=RUNNABLE, group=TGRP-TestSolrCoreProperties]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=41, name=qtp273092679-41 Acceptor2 SelectChannelConnector@0.0.0.0:63924, state=BLOCKED, group=TGRP-TestSolrCoreProperties]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=42, name=qtp273092679-42 Acceptor3 SelectChannelConnector@0.0.0.0:63924, state=BLOCKED, group=TGRP-TestSolrCoreProperties]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=39, name=qtp273092679-39 Acceptor0 SelectChannelConnector@0.0.0.0:63924, state=BLOCKED, group=TGRP-TestSolrCoreProperties]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.TestSolrCoreProperties: 
   1) Thread[id=40, name=qtp273092679-40 Acceptor1 SelectChannelConnector@0.0.0.0:63924, state=RUNNABLE, group=TGRP-TestSolrCoreProperties]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=41, name=qtp273092679-41 Acceptor2 SelectChannelConnector@0.0.0.0:63924, state=BLOCKED, group=TGRP-TestSolrCoreProperties]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=42, name=qtp273092679-42 Acceptor3 SelectChannelConnector@0.0.0.0:63924, state=BLOCKED, group=TGRP-TestSolrCoreProperties]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=39, name=qtp273092679-39 Acceptor0 SelectChannelConnector@0.0.0.0:63924, state=BLOCKED, group=TGRP-TestSolrCoreProperties]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([8ADC3589EAB97A9]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.TestSolrCoreProperties

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=40, name=qtp273092679-40 Acceptor1 SelectChannelConnector@0.0.0.0:63924, state=RUNNABLE, group=TGRP-TestSolrCoreProperties]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=41, name=qtp273092679-41 Acceptor2 SelectChannelConnector@0.0.0.0:63924, state=BLOCKED, group=TGRP-TestSolrCoreProperties]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=42, name=qtp273092679-42 Acceptor3 SelectChannelConnector@0.0.0.0:63924, state=BLOCKED, group=TGRP-TestSolrCoreProperties]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=39, name=qtp273092679-39 Acceptor0 SelectChannelConnector@0.0.0.0:63924, state=BLOCKED, group=TGRP-TestSolrCoreProperties]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=40, name=qtp273092679-40 Acceptor1 SelectChannelConnector@0.0.0.0:63924, state=RUNNABLE, group=TGRP-TestSolrCoreProperties]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=41, name=qtp273092679-41 Acceptor2 SelectChannelConnector@0.0.0.0:63924, state=BLOCKED, group=TGRP-TestSolrCoreProperties]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=42, name=qtp273092679-42 Acceptor3 SelectChannelConnector@0.0.0.0:63924, state=BLOCKED, group=TGRP-TestSolrCoreProperties]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=39, name=qtp273092679-39 Acceptor0 SelectChannelConnector@0.0.0.0:63924, state=BLOCKED, group=TGRP-TestSolrCoreProperties]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([8ADC3589EAB97A9]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.update.SolrCmdDistributorTest

Error Message:
20 threads leaked from SUITE scope at org.apache.solr.update.SolrCmdDistributorTest:     1) Thread[id=83, name=qtp326170431-83 Acceptor1 SelectChannelConnector@0.0.0.0:59797, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=40, name=qtp593385024-40 Acceptor0 SelectChannelConnector@0.0.0.0:38543, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=82, name=qtp326170431-82 Acceptor0 SelectChannelConnector@0.0.0.0:59797, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=43, name=qtp593385024-43 Acceptor3 SelectChannelConnector@0.0.0.0:38543, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=41, name=qtp593385024-41 Acceptor1 SelectChannelConnector@0.0.0.0:38543, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=85, name=qtp326170431-85 Acceptor3 SelectChannelConnector@0.0.0.0:59797, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=69, name=qtp1364030911-69 Acceptor1 SelectChannelConnector@0.0.0.0:59424, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=54, name=qtp907466781-54 Acceptor0 SelectChannelConnector@0.0.0.0:33240, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=71, name=qtp1364030911-71 Acceptor3 SelectChannelConnector@0.0.0.0:59424, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=28, name=qtp1484531491-28 Acceptor2 SelectChannelConnector@0.0.0.0:26902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=56, name=qtp907466781-56 Acceptor2 SelectChannelConnector@0.0.0.0:33240, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=26, name=qtp1484531491-26 Acceptor0 SelectChannelConnector@0.0.0.0:26902, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=29, name=qtp1484531491-29 Acceptor3 SelectChannelConnector@0.0.0.0:26902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=27, name=qtp1484531491-27 Acceptor1 SelectChannelConnector@0.0.0.0:26902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=42, name=qtp593385024-42 Acceptor2 SelectChannelConnector@0.0.0.0:38543, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=70, name=qtp1364030911-70 Acceptor2 SelectChannelConnector@0.0.0.0:59424, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=57, name=qtp907466781-57 Acceptor3 SelectChannelConnector@0.0.0.0:33240, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=68, name=qtp1364030911-68 Acceptor0 SelectChannelConnector@0.0.0.0:59424, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=84, name=qtp326170431-84 Acceptor2 SelectChannelConnector@0.0.0.0:59797, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=55, name=qtp907466781-55 Acceptor1 SelectChannelConnector@0.0.0.0:33240, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 20 threads leaked from SUITE scope at org.apache.solr.update.SolrCmdDistributorTest: 
   1) Thread[id=83, name=qtp326170431-83 Acceptor1 SelectChannelConnector@0.0.0.0:59797, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=40, name=qtp593385024-40 Acceptor0 SelectChannelConnector@0.0.0.0:38543, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=82, name=qtp326170431-82 Acceptor0 SelectChannelConnector@0.0.0.0:59797, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=43, name=qtp593385024-43 Acceptor3 SelectChannelConnector@0.0.0.0:38543, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=41, name=qtp593385024-41 Acceptor1 SelectChannelConnector@0.0.0.0:38543, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=85, name=qtp326170431-85 Acceptor3 SelectChannelConnector@0.0.0.0:59797, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=69, name=qtp1364030911-69 Acceptor1 SelectChannelConnector@0.0.0.0:59424, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=54, name=qtp907466781-54 Acceptor0 SelectChannelConnector@0.0.0.0:33240, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=71, name=qtp1364030911-71 Acceptor3 SelectChannelConnector@0.0.0.0:59424, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=28, name=qtp1484531491-28 Acceptor2 SelectChannelConnector@0.0.0.0:26902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=56, name=qtp907466781-56 Acceptor2 SelectChannelConnector@0.0.0.0:33240, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=26, name=qtp1484531491-26 Acceptor0 SelectChannelConnector@0.0.0.0:26902, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=29, name=qtp1484531491-29 Acceptor3 SelectChannelConnector@0.0.0.0:26902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=27, name=qtp1484531491-27 Acceptor1 SelectChannelConnector@0.0.0.0:26902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=42, name=qtp593385024-42 Acceptor2 SelectChannelConnector@0.0.0.0:38543, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=70, name=qtp1364030911-70 Acceptor2 SelectChannelConnector@0.0.0.0:59424, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=57, name=qtp907466781-57 Acceptor3 SelectChannelConnector@0.0.0.0:33240, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=68, name=qtp1364030911-68 Acceptor0 SelectChannelConnector@0.0.0.0:59424, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=84, name=qtp326170431-84 Acceptor2 SelectChannelConnector@0.0.0.0:59797, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=55, name=qtp907466781-55 Acceptor1 SelectChannelConnector@0.0.0.0:33240, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([8ADC3589EAB97A9]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.update.SolrCmdDistributorTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=40, name=qtp593385024-40 Acceptor0 SelectChannelConnector@0.0.0.0:38543, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=83, name=qtp326170431-83 Acceptor1 SelectChannelConnector@0.0.0.0:59797, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=82, name=qtp326170431-82 Acceptor0 SelectChannelConnector@0.0.0.0:59797, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=43, name=qtp593385024-43 Acceptor3 SelectChannelConnector@0.0.0.0:38543, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=41, name=qtp593385024-41 Acceptor1 SelectChannelConnector@0.0.0.0:38543, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=54, name=qtp907466781-54 Acceptor0 SelectChannelConnector@0.0.0.0:33240, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=69, name=qtp1364030911-69 Acceptor1 SelectChannelConnector@0.0.0.0:59424, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=85, name=qtp326170431-85 Acceptor3 SelectChannelConnector@0.0.0.0:59797, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=71, name=qtp1364030911-71 Acceptor3 SelectChannelConnector@0.0.0.0:59424, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=28, name=qtp1484531491-28 Acceptor2 SelectChannelConnector@0.0.0.0:26902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=26, name=qtp1484531491-26 Acceptor0 SelectChannelConnector@0.0.0.0:26902, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=56, name=qtp907466781-56 Acceptor2 SelectChannelConnector@0.0.0.0:33240, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=29, name=qtp1484531491-29 Acceptor3 SelectChannelConnector@0.0.0.0:26902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=27, name=qtp1484531491-27 Acceptor1 SelectChannelConnector@0.0.0.0:26902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=42, name=qtp593385024-42 Acceptor2 SelectChannelConnector@0.0.0.0:38543, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=70, name=qtp1364030911-70 Acceptor2 SelectChannelConnector@0.0.0.0:59424, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=57, name=qtp907466781-57 Acceptor3 SelectChannelConnector@0.0.0.0:33240, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=68, name=qtp1364030911-68 Acceptor0 SelectChannelConnector@0.0.0.0:59424, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=84, name=qtp326170431-84 Acceptor2 SelectChannelConnector@0.0.0.0:59797, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=55, name=qtp907466781-55 Acceptor1 SelectChannelConnector@0.0.0.0:33240, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=40, name=qtp593385024-40 Acceptor0 SelectChannelConnector@0.0.0.0:38543, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=83, name=qtp326170431-83 Acceptor1 SelectChannelConnector@0.0.0.0:59797, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=82, name=qtp326170431-82 Acceptor0 SelectChannelConnector@0.0.0.0:59797, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=43, name=qtp593385024-43 Acceptor3 SelectChannelConnector@0.0.0.0:38543, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=41, name=qtp593385024-41 Acceptor1 SelectChannelConnector@0.0.0.0:38543, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=54, name=qtp907466781-54 Acceptor0 SelectChannelConnector@0.0.0.0:33240, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=69, name=qtp1364030911-69 Acceptor1 SelectChannelConnector@0.0.0.0:59424, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=85, name=qtp326170431-85 Acceptor3 SelectChannelConnector@0.0.0.0:59797, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=71, name=qtp1364030911-71 Acceptor3 SelectChannelConnector@0.0.0.0:59424, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=28, name=qtp1484531491-28 Acceptor2 SelectChannelConnector@0.0.0.0:26902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=26, name=qtp1484531491-26 Acceptor0 SelectChannelConnector@0.0.0.0:26902, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=56, name=qtp907466781-56 Acceptor2 SelectChannelConnector@0.0.0.0:33240, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=29, name=qtp1484531491-29 Acceptor3 SelectChannelConnector@0.0.0.0:26902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=27, name=qtp1484531491-27 Acceptor1 SelectChannelConnector@0.0.0.0:26902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=42, name=qtp593385024-42 Acceptor2 SelectChannelConnector@0.0.0.0:38543, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=70, name=qtp1364030911-70 Acceptor2 SelectChannelConnector@0.0.0.0:59424, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=57, name=qtp907466781-57 Acceptor3 SelectChannelConnector@0.0.0.0:33240, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=68, name=qtp1364030911-68 Acceptor0 SelectChannelConnector@0.0.0.0:59424, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=84, name=qtp326170431-84 Acceptor2 SelectChannelConnector@0.0.0.0:59797, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=55, name=qtp907466781-55 Acceptor1 SelectChannelConnector@0.0.0.0:33240, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([8ADC3589EAB97A9]:0)




Build Log:
[...truncated 8311 lines...]
[junit4:junit4] Suite: org.apache.solr.TestSolrCoreProperties
[junit4:junit4]   2> 1448 T32 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 1893 T32 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:63924
[junit4:junit4]   2> 1965 T32 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1966 T32 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.TestSolrCoreProperties-1351782850760
[junit4:junit4]   2> 1966 T32 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: './org.apache.solr.TestSolrCoreProperties-1351782850760/'
[junit4:junit4]   2> 2013 T32 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 2014 T32 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2015 T32 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.TestSolrCoreProperties-1351782850760
[junit4:junit4]   2> 2015 T32 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.TestSolrCoreProperties-1351782850760/solr.xml
[junit4:junit4]   2> 2016 T32 oasc.CoreContainer.<init> New CoreContainer 1264124060
[junit4:junit4]   2> 2018 T32 oasc.CoreContainer$Initializer.initialize no solr.xml file found - using default
[junit4:junit4]   2> 2019 T32 oasc.CoreContainer.load Loading CoreContainer using Solr Home: './org.apache.solr.TestSolrCoreProperties-1351782850760/'
[junit4:junit4]   2> 2019 T32 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.TestSolrCoreProperties-1351782850760/'
[junit4:junit4]   2> 2060 T32 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 2110 T32 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.TestSolrCoreProperties-1351782850760/collection1
[junit4:junit4]   2> 2112 T32 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.TestSolrCoreProperties-1351782850760/collection1/'
[junit4:junit4]   2> 2150 T32 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 2200 T32 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 2202 T32 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 2206 T32 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 2226 T32 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 2228 T32 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.TestSolrCoreProperties-1351782850760/collection1/, dataDir=./org.apache.solr.TestSolrCoreProperties-1351782850760/collection1/data/
[junit4:junit4]   2> 2228 T32 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 2229 T32 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.TestSolrCoreProperties-1351782850760/collection1/data/index/
[junit4:junit4]   2> 2230 T32 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.TestSolrCoreProperties-1351782850760/collection1/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2231 T32 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.TestSolrCoreProperties-1351782850760/collection1/data/index forceNew:false
[junit4:junit4]   2> 2233 T32 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2165615c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@fcc10a7),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 2233 T32 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 2234 T32 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2235 T32 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 2235 T32 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 2236 T32 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 2236 T32 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2253 T32 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2256 T32 oass.SolrIndexSearcher.<init> Opening Searcher@4c45a3ad main
[junit4:junit4]   2> 2256 T32 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 2257 T32 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2257 T32 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2258 T32 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 2258 T32 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 2259 T32 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 2259 T32 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 2260 T32 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 2260 T32 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 2261 T32 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 2261 T32 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 2262 T32 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 2262 T32 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 2276 T32 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 2276 T43 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4c45a3ad main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2276 T32 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 2278 T32 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 2360 T32 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C2 name=collection1 org.apache.solr.core.SolrCore@6e4d4b9d
[junit4:junit4]   2> 3660 T45 C2 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=0 status=0 QTime=107 
[junit4:junit4]   2> 3903 T32 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1264124060
[junit4:junit4]   2> 3903 T32 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@6e4d4b9d
[junit4:junit4]   2> 3904 T32 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 3905 T32 oasc.SolrCore.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 3905 T32 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 3906 T32 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 3906 T32 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 3908 T32 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 4010 T32 oejut.QueuedThreadPool.doStop WARNING 4 threads could not be stopped
[junit4:junit4]   2> 4020 T31 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 4 leaked thread(s).
[junit4:junit4]   2> 24108 T31 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 4 threads leaked from SUITE scope at org.apache.solr.TestSolrCoreProperties: 
[junit4:junit4]   2> 	   1) Thread[id=40, name=qtp273092679-40 Acceptor1 SelectChannelConnector@0.0.0.0:63924, state=RUNNABLE, group=TGRP-TestSolrCoreProperties]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   2) Thread[id=41, name=qtp273092679-41 Acceptor2 SelectChannelConnector@0.0.0.0:63924, state=BLOCKED, group=TGRP-TestSolrCoreProperties]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   3) Thread[id=42, name=qtp273092679-42 Acceptor3 SelectChannelConnector@0.0.0.0:63924, state=BLOCKED, group=TGRP-TestSolrCoreProperties]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   4) Thread[id=39, name=qtp273092679-39 Acceptor0 SelectChannelConnector@0.0.0.0:63924, state=BLOCKED, group=TGRP-TestSolrCoreProperties]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 24110 T31 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
[junit4:junit4]   2> 	   1) Thread[id=40, name=qtp273092679-40 Acceptor1 SelectChannelConnector@0.0.0.0:63924, state=RUNNABLE, group=TGRP-TestSolrCoreProperties]
[junit4:junit4]   2> 	   2) Thread[id=41, name=qtp273092679-41 Acceptor2 SelectChannelConnector@0.0.0.0:63924, state=BLOCKED, group=TGRP-TestSolrCoreProperties]
[junit4:junit4]   2> 	   3) Thread[id=42, name=qtp273092679-42 Acceptor3 SelectChannelConnector@0.0.0.0:63924, state=BLOCKED, group=TGRP-TestSolrCoreProperties]
[junit4:junit4]   2> 	   4) Thread[id=39, name=qtp273092679-39 Acceptor0 SelectChannelConnector@0.0.0.0:63924, state=BLOCKED, group=TGRP-TestSolrCoreProperties]
[junit4:junit4]   2> 27153 T31 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
[junit4:junit4]   2> 	   1) Thread[id=40, name=qtp273092679-40 Acceptor1 SelectChannelConnector@0.0.0.0:63924, state=RUNNABLE, group=TGRP-TestSolrCoreProperties]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   2) Thread[id=41, name=qtp273092679-41 Acceptor2 SelectChannelConnector@0.0.0.0:63924, state=BLOCKED, group=TGRP-TestSolrCoreProperties]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   3) Thread[id=42, name=qtp273092679-42 Acceptor3 SelectChannelConnector@0.0.0.0:63924, state=BLOCKED, group=TGRP-TestSolrCoreProperties]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   4) Thread[id=39, name=qtp273092679-39 Acceptor0 SelectChannelConnector@0.0.0.0:63924, state=BLOCKED, group=TGRP-TestSolrCoreProperties]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene41: {}, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=ko, timezone=Pacific/Funafuti
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=5,free=107408744,total=131072000
[junit4:junit4]   2> NOTE: All tests run in this JVM: [UpdateParamsTest, PingRequestHandlerTest, IndexReaderFactoryTest, TestPropInject, DocumentBuilderTest, TestFoldingMultitermQuery, SolrIndexConfigTest, TestSolrCoreProperties]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSolrCoreProperties -Dtests.seed=8ADC3589EAB97A9 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ko -Dtests.timezone=Pacific/Funafuti -Dtests.file.encoding=UTF-8
[junit4:junit4] ERROR   0.00s J1 | TestSolrCoreProperties (suite) <<<
[junit4:junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.TestSolrCoreProperties: 
[junit4:junit4]    >    1) Thread[id=40, name=qtp273092679-40 Acceptor1 SelectChannelConnector@0.0.0.0:63924, state=RUNNABLE, group=TGRP-TestSolrCoreProperties]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=41, name=qtp273092679-41 Acceptor2 SelectChannelConnector@0.0.0.0:63924, state=BLOCKED, group=TGRP-TestSolrCoreProperties]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=42, name=qtp273092679-42 Acceptor3 SelectChannelConnector@0.0.0.0:63924, state=BLOCKED, group=TGRP-TestSolrCoreProperties]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=39, name=qtp273092679-39 Acceptor0 SelectChannelConnector@0.0.0.0:63924, state=BLOCKED, group=TGRP-TestSolrCoreProperties]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([8ADC3589EAB97A9]:0)
[junit4:junit4]    > Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4]    >    1) Thread[id=40, name=qtp273092679-40 Acceptor1 SelectChannelConnector@0.0.0.0:63924, state=RUNNABLE, group=TGRP-TestSolrCoreProperties]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=41, name=qtp273092679-41 Acceptor2 SelectChannelConnector@0.0.0.0:63924, state=BLOCKED, group=TGRP-TestSolrCoreProperties]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=42, name=qtp273092679-42 Acceptor3 SelectChannelConnector@0.0.0.0:63924, state=BLOCKED, group=TGRP-TestSolrCoreProperties]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=39, name=qtp273092679-39 Acceptor0 SelectChannelConnector@0.0.0.0:63924, state=BLOCKED, group=TGRP-TestSolrCoreProperties]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([8ADC3589EAB97A9]:0)
[junit4:junit4] Completed on J1 in 26.50s, 1 test, 2 errors <<< FAILURES!

[...truncated 157 lines...]
[junit4:junit4] Suite: org.apache.solr.update.SolrCmdDistributorTest
[junit4:junit4]   2> 5 T19 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> 516 T19 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 949 T19 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:26902
[junit4:junit4]   2> 1015 T19 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1016 T19 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 1016 T19 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 1073 T19 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1074 T19 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1075 T19 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 1075 T19 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 1076 T19 oasc.CoreContainer.<init> New CoreContainer 1384218227
[junit4:junit4]   2> 1077 T19 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 1077 T19 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 1124 T19 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 1184 T19 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 1191 T19 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 1193 T19 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 1194 T19 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 1246 T19 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 1325 T19 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 1327 T19 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 1383 T19 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 2184 T19 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 2203 T19 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 2207 T19 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 2233 T19 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2240 T19 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2246 T19 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2250 T19 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2254 T19 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1351782844925/control/data/
[junit4:junit4]   2> 2255 T19 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 2256 T19 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1351782844925/control/data/index/
[junit4:junit4]   2> 2257 T19 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1351782844925/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2258 T19 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1351782844925/control/data/index forceNew:false
[junit4:junit4]   2> 2263 T19 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@25bea571 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b625a2d),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 2263 T19 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 2265 T19 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2266 T19 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2267 T19 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2267 T19 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2281 T19 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2288 T19 oass.SolrIndexSearcher.<init> Opening Searcher@7acfc79f main
[junit4:junit4]   2> 2288 T19 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 2320 T19 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2321 T19 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2322 T19 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 2323 T19 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 2323 T19 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 2324 T19 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 2325 T19 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 2325 T19 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 2326 T19 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 2327 T19 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 2327 T19 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 2328 T19 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 2346 T34 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7acfc79f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2350 T19 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 2351 T19 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 2352 T19 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 2419 T19 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 2443 T19 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 2447 T19 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:38543
[junit4:junit4]   2> 2448 T19 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2449 T19 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 2449 T19 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2493 T19 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 2494 T19 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2494 T19 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 2495 T19 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 2496 T19 oasc.CoreContainer.<init> New CoreContainer 1612005438
[junit4:junit4]   2> 2497 T19 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2497 T19 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2530 T19 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 2552 T19 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 2553 T19 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 2555 T19 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 2556 T19 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 2604 T19 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 2661 T19 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 2662 T19 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 2707 T19 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 3241 T19 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 3254 T19 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 3258 T19 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 3276 T19 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 3281 T19 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 3285 T19 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 3288 T19 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 3290 T19 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1351782844925/shard0/data/
[junit4:junit4]   2> 3291 T19 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 3292 T19 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1351782844925/shard0/data/index/
[junit4:junit4]   2> 3293 T19 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1351782844925/shard0/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 3294 T19 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1351782844925/shard0/data/index forceNew:false
[junit4:junit4]   2> 3297 T19 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@60d98f23 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5b3cf82b),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 3298 T19 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 3299 T19 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 3300 T19 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 3300 T19 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 3301 T19 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 3312 T19 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 3317 T19 oass.SolrIndexSearcher.<init> Opening Searcher@12fd41e0 main
[junit4:junit4]   2> 3318 T19 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3319 T19 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 3320 T19 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 3320 T19 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 3321 T19 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 3321 T19 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 3322 T19 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 3322 T19 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 3323 T19 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 3323 T19 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 3324 T19 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 3324 T19 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 3325 T19 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 3339 T48 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@12fd41e0 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 3338 T19 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 3340 T19 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 3340 T19 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3341 T19 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3352 T19 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 3354 T19 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:33240
[junit4:junit4]   2> 3356 T19 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3356 T19 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 3356 T19 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 3388 T19 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 3388 T19 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3389 T19 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 3389 T19 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 3390 T19 oasc.CoreContainer.<init> New CoreContainer 557552037
[junit4:junit4]   2> 3390 T19 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 3391 T19 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 3417 T19 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 3440 T19 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 3441 T19 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 3443 T19 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 3444 T19 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 3481 T19 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 3523 T19 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 3524 T19 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 3550 T19 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 4105 T19 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 4118 T19 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 4121 T19 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 4138 T19 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4142 T19 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4147 T19 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4149 T19 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4152 T19 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1351782844925/shard1/data/
[junit4:junit4]   2> 4152 T19 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 4153 T19 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1351782844925/shard1/data/index/
[junit4:junit4]   2> 4154 T19 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1351782844925/shard1/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 4155 T19 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1351782844925/shard1/data/index forceNew:false
[junit4:junit4]   2> 4158 T19 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4641e51b lockFactory=org.apache.lucene.store.NativeFSLockFactory@4fc09f93),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 4159 T19 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 4160 T19 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 4161 T19 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 4161 T19 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 4162 T19 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 4171 T19 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 4176 T19 oass.SolrIndexSearcher.<init> Opening Searcher@6caa873c main
[junit4:junit4]   2> 4176 T19 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 4177 T19 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 4178 T19 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 4179 T19 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 4179 T19 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4179 T19 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 4180 T19 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4180 T19 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4181 T19 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4181 T19 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4182 T19 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4182 T19 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4183 T19 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 4196 T19 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 4196 T62 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6caa873c main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 4196 T19 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 4197 T19 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 4198 T19 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 4208 T19 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 4210 T19 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:59424
[junit4:junit4]   2> 4211 T19 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4212 T19 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 4212 T19 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4241 T19 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 4241 T19 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4242 T19 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 4242 T19 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 4243 T19 oasc.CoreContainer.<init> New CoreContainer 1535633470
[junit4:junit4]   2> 4243 T19 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4244 T19 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4276 T19 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4293 T19 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 4294 T19 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 4295 T19 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4296 T19 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 4332 T19 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 4372 T19 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 4374 T19 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4392 T19 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 4887 T19 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 4907 T19 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 4912 T19 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 4932 T19 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4939 T19 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4949 T19 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4952 T19 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4954 T19 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1351782844925/shard2/data/
[junit4:junit4]   2> 4954 T19 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 4955 T19 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1351782844925/shard2/data/index/
[junit4:junit4]   2> 4956 T19 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1351782844925/shard2/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 4961 T19 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1351782844925/shard2/data/index forceNew:false
[junit4:junit4]   2> 4964 T19 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@31f8ee8b lockFactory=org.apache.lucene.store.NativeFSLockFactory@ba33c71),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 4964 T19 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 4965 T19 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 4966 T19 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 4967 T19 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 4967 T19 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 4975 T19 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 4980 T19 oass.SolrIndexSearcher.<init> Opening Searcher@4be9246f main
[junit4:junit4]   2> 4981 T19 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 4982 T19 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 4983 T19 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 4984 T19 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 4985 T19 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4985 T19 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 4986 T19 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4987 T19 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4988 T19 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4988 T19 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4989 T19 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4990 T19 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4990 T19 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 5004 T76 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4be9246f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5004 T19 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 5005 T19 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 5006 T19 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 5006 T19 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 5016 T19 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 5019 T19 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:59797
[junit4:junit4]   2> 5020 T19 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 5020 T19 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 5021 T19 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 5055 T19 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 5055 T19 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 5056 T19 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 5056 T19 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 5057 T19 oasc.CoreContainer.<init> New CoreContainer 1116277913
[junit4:junit4]   2> 5058 T19 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 5058 T19 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 5086 T19 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 5103 T19 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 5104 T19 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 5106 T19 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 5106 T19 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 5144 T19 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 5184 T19 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 5185 T19 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 5203 T19 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 5805 T19 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 5818 T19 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 5822 T19 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5839 T19 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5844 T19 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5849 T19 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5851 T19 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5853 T19 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1351782844925/shard3/data/
[junit4:junit4]   2> 5853 T19 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 5855 T19 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1351782844925/shard3/data/index/
[junit4:junit4]   2> 5856 T19 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1351782844925/shard3/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5856 T19 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1351782844925/shard3/data/index forceNew:false
[junit4:junit4]   2> 5861 T19 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@12160823 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4389d21),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5862 T19 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 5863 T19 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 5864 T19 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 5864 T19 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 5865 T19 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 5880 T19 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 5888 T19 oass.SolrIndexSearcher.<init> Opening Searcher@4e1f11e4 main
[junit4:junit4]   2> 5888 T19 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 5890 T19 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 5890 T19 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 5891 T19 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 5891 T19 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 5892 T19 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 5893 T19 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 5893 T19 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 5893 T19 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 5894 T19 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 5894 T19 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 5895 T19 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 5895 T19 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 5909 T89 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4e1f11e4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5909 T19 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 5910 T19 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 5911 T19 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 5911 T19 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C2 name=collection1 org.apache.solr.core.SolrCore@2cbf8e01
[junit4:junit4]   2> 7168 T33 C2 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@25bea571 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b625a2d),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 7169 T33 C2 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 7208 T33 C2 oass.SolrIndexSearcher.<init> Opening Searcher@52fda209 realtime
[junit4:junit4]   2> 7209 T33 C2 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 7214 T33 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1417447055879569408)} 0 109
[junit4:junit4]   2> ASYNC  NEW_CORE C3 name=collection1 org.apache.solr.core.SolrCore@107418be
[junit4:junit4]   2> 7393 T45 C3 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@60d98f23 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5b3cf82b),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 7394 T45 C3 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 7396 T45 C3 oass.SolrIndexSearcher.<init> Opening Searcher@142aeb8a realtime
[junit4:junit4]   2> 7396 T45 C3 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 7397 T45 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1417447056142761984)} 0 8
[junit4:junit4]   2> ASYNC  NEW_CORE C4 name=collection1 org.apache.solr.core.SolrCore@456964ea
[junit4:junit4]   2> 7409 T61 C4 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4641e51b lockFactory=org.apache.lucene.store.NativeFSLockFactory@4fc09f93),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 7410 T61 C4 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 7412 T61 C4 oass.SolrIndexSearcher.<init> Opening Searcher@18a76cea realtime
[junit4:junit4]   2> 7413 T61 C4 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 7413 T61 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1417447056160587776)} 0 8
[junit4:junit4]   2> ASYNC  NEW_CORE C5 name=collection1 org.apache.solr.core.SolrCore@7db6306b
[junit4:junit4]   2> 7422 T74 C5 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@31f8ee8b lockFactory=org.apache.lucene.store.NativeFSLockFactory@ba33c71),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 7423 T74 C5 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 7424 T74 C5 oass.SolrIndexSearcher.<init> Opening Searcher@5c4e3bb0 realtime
[junit4:junit4]   2> 7425 T74 C5 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 7425 T74 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1417447056175267840)} 0 5
[junit4:junit4]   2> ASYNC  NEW_CORE C6 name=collection1 org.apache.solr.core.SolrCore@10fe5966
[junit4:junit4]   2> 7436 T88 C6 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@12160823 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4389d21),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 7437 T88 C6 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 7440 T88 C6 oass.SolrIndexSearcher.<init> Opening Searcher@3a7c4526 realtime
[junit4:junit4]   2> 7440 T88 C6 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 7441 T88 C6 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1417447056188899328)} 0 8
[junit4:junit4]   2> 7469 T19 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16
[junit4:junit4]   2> 7564 T31 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1 (1417447056323117056)]} 0 11
[junit4:junit4]   2> 7574 T19 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:26902/solr/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4]   2> 7598 T33 C2 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 7601 T33 C2 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@25bea571 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b625a2d),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@25bea571 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b625a2d),segFN=segments_2,generation=2,filenames=[_0.tis, _0.nrm, _0.fnm, _0.tii, _0.frq, segments_2, _0.fdx, _0.si, _0.prx, _0.fdt]
[junit4:junit4]   2> 7601 T33 C2 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 7607 T33 C2 oass.SolrIndexSearcher.<init> Opening Searcher@35d5551e main
[junit4:junit4]   2> 7607 T33 C2 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 7608 T34 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@35d5551e main{StandardDirectoryReader(segments_2:3:nrt _0(4.1):C1)}
[junit4:junit4]   2> 7608 T33 C2 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 7609 T33 C2 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&commit_end_point=true&expungeDeletes=false&wt=javabin&softCommit=false&version=2} {commit=} 0 11
[junit4:junit4]   2> 7740 T32 C2 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=1 status=0 QTime=83 
[junit4:junit4]   2> 7751 T33 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&commit_end_point=true&version=2} {add=[2 (1417447056517103616), 3 (1417447056520249344)]} 0 5
[junit4:junit4]   2> 7753 T47 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&commit_end_point=true&version=2} {add=[2 (1417447056517103616), 3 (1417447056520249344), 4 (1417447056522346496)]} 0 7
[junit4:junit4]   2> 7755 T19 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:26902/solr/, StdNode: http://127.0.0.1:38543/solr/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4]   2> 7759 T46 C3 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 7759 T31 C2 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 7763 T46 C3 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@60d98f23 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5b3cf82b),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@60d98f23 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5b3cf82b),segFN=segments_2,generation=2,filenames=[_0.tis, _0.nrm, _0.fnm, _0.tii, _0.frq, segments_2, _0.fdx, _0.si, _0.prx, _0.fdt]
[junit4:junit4]   2> 7764 T46 C3 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 7766 T46 C3 oass.SolrIndexSearcher.<init> Opening Searcher@75fa1077 main
[junit4:junit4]   2> 7766 T46 C3 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 7767 T46 C3 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 7767 T48 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@75fa1077 main{StandardDirectoryReader(segments_2:3:nrt _0(4.1):C3)}
[junit4:junit4]   2> 7768 T46 C3 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&commit_end_point=true&expungeDeletes=false&wt=javabin&softCommit=false&version=2} {commit=} 0 9
[junit4:junit4]   2> 7830 T31 C2 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@25bea571 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b625a2d),segFN=segments_2,generation=2,filenames=[_0.tis, _0.nrm, _0.fnm, _0.tii, _0.frq, segments_2, _0.fdx, _0.si, _0.prx, _0.fdt]
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@25bea571 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b625a2d),segFN=segments_3,generation=3,filenames=[_0.nrm, _0.tis, _1.tis, _0.fnm, _1.frq, _1.fnm, _1.prx, _1.fdx, _0.tii, _1.fdt, _1.si, _0.frq, _1.tii, _0.fdx, _0.si, _0.prx, _1.nrm, segments_3, _0.fdt]
[junit4:junit4]   2> 7830 T31 C2 oasc.SolrDeletionPolicy.updateCommits newest commit = 3
[junit4:junit4]   2> 7840 T31 C2 oass.SolrIndexSearcher.<init> Opening Searcher@76ca681e main
[junit4:junit4]   2> 7840 T31 C2 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 7841 T34 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@76ca681e main{StandardDirectoryReader(segments_3:5:nrt _0(4.1):C1 _1(4.1):C2)}
[junit4:junit4]   2> 7842 T31 C2 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 7843 T31 C2 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&commit_end_point=true&expungeDeletes=false&wt=javabin&softCommit=false&version=2} {commit=} 0 84
[junit4:junit4]   2> 7847 T32 C2 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=3 status=0 QTime=1 
[junit4:junit4]   2> 7855 T47 C3 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=3 status=0 QTime=0 
[junit4:junit4]   2> 7868 T46 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&commit_end_point=true&version=2} {delete=[2 (-1417447056643981312)]} 0 1
[junit4:junit4]   2> 7869 T31 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&commit_end_point=true&version=2} {delete=[2 (-1417447056643981312)]} 0 2
[junit4:junit4]   2> 7870 T19 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:26902/solr/, StdNode: http://127.0.0.1:38543/solr/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4]   2> 7875 T33 C2 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 7876 T45 C3 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 7883 T33 C2 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@25bea571 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b625a2d),segFN=segments_3,generation=3,filenames=[_0.nrm, _0.tis, _1.tis, _0.fnm, _1.frq, _1.fnm, _1.prx, _1.fdx, _0.tii, _1.fdt, _1.si, _0.frq, _1.tii, _0.fdx, _0.si, _0.prx, _1.nrm, segments_3, _0.fdt]
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@25bea571 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b625a2d),segFN=segments_4,generation=4,filenames=[_0.nrm, _0.tis, _1.tis, _0.fnm, _1.frq, _1_1.del, _1.fnm, _1.prx, _1.fdx, _0.tii, _1.fdt, _1.si, _0.frq, _1.tii, _0.fdx, _0.si, _0.prx, _1.nrm, _0.fdt, segments_4]
[junit4:junit4]   2> 7884 T45 C3 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@60d98f23 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5b3cf82b),segFN=segments_2,generation=2,filenames=[_0.tis, _0.nrm, _0.fnm, _0.tii, _0.frq, segments_2, _0.fdx, _0.si, _0.prx, _0.fdt]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@60d98f23 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5b3cf82b),segFN=segments_3,generation=3,filenames=[_0.tis, _0.nrm, _0.fnm, _0.tii, _0.frq, _0_1.del, _0.fdx, _0.si, _0.prx, _0.fdt, segments_3]
[junit4:junit4]   2> 7884 T33 C2 oasc.SolrDeletionPolicy.updateCommits newest commit = 4
[junit4:junit4]   2> 7885 T45 C3 oasc.SolrDeletionPolicy.updateCommits newest commit = 3
[junit4:junit4]   2> 7886 T45 C3 oass.SolrIndexSearcher.<init> Opening Searcher@2e3ed883 main
[junit4:junit4]   2> 7886 T45 C3 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 7888 T48 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2e3ed883 main{StandardDirectoryReader(segments_3:4:nrt _0(4.1):C3/1)}
[junit4:junit4]   2> 7888 T45 C3 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 7889 T45 C3 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&commit_end_point=true&expungeDeletes=false&wt=javabin&softCommit=false&version=2} {commit=} 0 14
[junit4:junit4]   2> 7889 T33 C2 oass.SolrIndexSearcher.<init> Opening Searcher@5c2d8c20 main
[junit4:junit4]   2> 7890 T33 C2 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 7891 T34 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5c2d8c20 main{StandardDirectoryReader(segments_4:6:nrt _0(4.1):C1 _1(4.1):C2/1)}
[junit4:junit4]   2> 7891 T33 C2 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 7892 T33 C2 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&commit_end_point=true&expungeDeletes=false&wt=javabin&softCommit=false&version=2} {commit=} 0 17
[junit4:junit4]   2> 7896 T32 C2 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=2 status=0 QTime=1 
[junit4:junit4]   2> 7901 T47 C3 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=2 status=0 QTime=1 
[junit4:junit4]   2> 7906 T46 C3 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 8055 T46 C3 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@60d98f23 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5b3cf82b),segFN=segments_3,generation=3,filenames=[_0.tis, _0.nrm, _0.fnm, _0.tii, _0.frq, _0_1.del, _0.fdx, _0.si, _0.prx, _0.fdt, segments_3]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@60d98f23 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5b3cf82b),segFN=segments_4,generation=4,filenames=[_1.fdx, _1.prx, _1.tis, _1.frq, _1.fdt, _1.si, _1.tii, _1.fnm, _1.nrm, segments_4]
[junit4:junit4]   2> 8056 T46 C3 oasc.SolrDeletionPolicy.updateCommits newest commit = 4
[junit4:junit4]   2> 8062 T46 C3 oass.SolrIndexSearcher.<init> Opening Searcher@6a05fdf main
[junit4:junit4]   2> 8062 T46 C3 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 8064 T48 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6a05fdf main{StandardDirectoryReader(segments_4:6:nrt _1(4.1):C2)}
[junit4:junit4]   2> 8064 T46 C3 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 8065 T46 C3 UPDATE [collection1] webapp=/solr path=/update params={optimize=true&maxSegments=1&waitSearcher=true&wt=javabin&version=2} {optimize=} 0 159
[junit4:junit4]   2> 8070 T61 C4 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 8072 T61 C4 oass.SolrIndexSearcher.<init> Opening Searcher@6c83396f main
[junit4:junit4]   2> 8073 T61 C4 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 8073 T62 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6c83396f main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4]   2> 8073 T61 C4 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 8074 T61 C4 UPDATE [collection1] webapp=/solr path=/update params={optimize=true&maxSegments=1&waitSearcher=true&wt=javabin&version=2} {optimize=} 0 4
[junit4:junit4]   2> 8078 T75 C5 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 8079 T75 C5 oass.SolrIndexSearcher.<init> Opening Searcher@19c9e8e8 main
[junit4:junit4]   2> 8079 T75 C5 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 8080 T76 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@19c9e8e8 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4]   2> 8080 T75 C5 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 8081 T75 C5 UPDATE [collection1] webapp=/solr path=/update params={optimize=true&maxSegments=1&waitSearcher=true&wt=javabin&version=2} {optimize=} 0 3
[junit4:junit4]   2> 8085 T88 C6 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 8086 T88 C6 oass.SolrIndexSearcher.<init> Opening Searcher@e196fe0 main
[junit4:junit4]   2> 8087 T88 C6 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 8088 T89 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@e196fe0 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4]   2> 8088 T88 C6 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 8089 T88 C6 UPDATE [collection1] webapp=/solr path=/update params={optimize=true&maxSegments=1&waitSearcher=true&wt=javabin&version=2} {optimize=} 0 3
[junit4:junit4]   2> 8178 T60 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[6 (1417447056882008064), 7 (1417447056902979584), 10 (1417447056914513920), 11 (1417447056919756800), 12 (1417447056924999680), 14 (1417447056929193984), 15 (1417447056935485441), 19 (1417447056942825473), 20 (1417447056954359808), 22 (1417447056961699840)]} 0 84
[junit4:junit4]   2> 8190 T61 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[46 (1417447056905076736), 49 (1417447056915562496), 52 (1417447056919756801), 53 (1417447056923951104), 55 (1417447056926048256), 62 (1417447056932339713), 63 (1417447056935485440), 64 (1417447056941776896), 65 (1417447056955408384), 66 (1417447056960651264)]} 0 74
[junit4:junit4]   2> 8191 T59 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[24 (1417447056898785280), 26 (1417447056904028160), 28 (1417447056916611072), 29 (1417447056922902528), 31 (1417447056927096832), 36 (1417447056932339712), 38 (1417447056937582592), 40 (1417447056949116928), 41 (1417447056958554112), 42 (1417447056964845568)]} 0 81
[junit4:junit4]   2> 8192 T73 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[8 (1417447056883056640), 9 (1417447056905076736), 10 (1417447056919756800), 17 (1417447056927096832), 19 (1417447056935485440), 21 (1417447056942825474), 22 (1417447056949116928), 23 (1417447056957505537), 25 (1417447056961699841), 27 (1417447056966942720)]} 0 97
[junit4:junit4]   2> 8192 T75 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[45 (1417447056914513920), 47 (1417447056927096834), 52 (1417447056933388288), 55 (1417447056939679745), 56 (1417447056944922624), 60 (1417447056954359808), 63 (1417447056959602688), 64 (1417447056964845569), 65 (1417447056973234176), 67 (1417447056976379904)]} 0 68
[junit4:junit4]   2> 8194 T45 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[6 (1417447056884105216), 13 (1417447056905076736), 14 (1417447056922902528), 15 (1417447056929193984), 16 (1417447056935485440), 18 (1417447056942825472), 19 (1417447056952262657), 20 (1417447056956456960), 23 (1417447056961699840), 24 (1417447056980574208)]} 0 99
[junit4:junit4]   2> 8216 T86 C6 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[10 (1417447056897736704), 11 (1417447056914513920), 20 (1417447056922902529), 22 (1417447056929193984), 26 (1417447056935485440), 27 (1417447056942825472), 28 (1417447056956456960), 29 (1417447056981622784), 30 (1417447056984768512), 32 (1417447056990011393)]} 0 120
[junit4:junit4]   2> 8224 T119 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[90 (1417447056939679744), 93 (1417447056942825472), 94 (1417447056947019776), 95 (1417447056955408385), 99 (1417447056964845570), 103 (1417447056978477056), 104 (1417447056986865664), 105 (1417447056991059969), 106 (1417447057012031488), 108 (1417447057015177217)]} 0 76
[junit4:junit4]   2> 8224 T106 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[68 (1417447056919756801), 69 (1417447056927096833), 71 (1417447056931291137), 73 (1417447056935485442), 74 (1417447056942825473), 77 (1417447056948068352), 79 (1417447056958554112), 80 (1417447056963796992), 81 (1417447056986865664), 84 (1417447056992108544)]} 0 94
[junit4:junit4]   2> 8225 T123 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[104 (1417447056943874048), 109 (1417447056954359809), 112 (1417447056957505538), 113 (1417447056961699840), 114 (1417447056964845568), 115 (1417447056966942721), 116 (1417447056988962817), 117 (1417447056992108545), 118 (1417447057014128640), 119 (1417447057016225794)]} 0 73
[junit4:junit4]   2> 8228 T108 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[68 (1417447056923951105), 73 (1417447056930242560), 75 (1417447056938631168), 80 (1417447056948068352), 81 (1417447056957505536), 82 (1417447056964845569), 83 (1417447056982671360), 84 (1417447056990011392), 88 (1417447057017274368), 89 (1417447057019371521)]} 0 94
[junit4:junit4]   2> 8229 T112 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[73 (1417447056930242561), 74 (1417447056936534016), 75 (1417447056948068352), 78 (1417447056961699841), 82 (1417447056966942720), 84 (1417447056981622786), 88 (1417447056990011392), 89 (1417447056992108544), 93 (1417447057012031488), 94 (1417447057014128641)]} 0 89
[junit4:junit4]   2> 8231 T110 C6 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[80 (1417447056928145408), 82 (1417447056935485441), 83 (1417447056942825473), 85 (1417447056949116929), 86 (1417447056954359808), 87 (1417447056961699840), 88 (1417447056979525632), 89 (1417447056981622785), 91 (1417447056990011392), 92 (1417447056992108545)]} 0 94
[junit4:junit4]   2> 8231 T47 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[48 (1417447056918708224), 52 (1417447056922902529), 57 (1417447056927096832), 58 (1417447056930242560), 59 (1417447056932339712), 61 (1417447056937582592), 62 (1417447056940728320), 65 (1417447056944922624), 69 (1417447056956456962), 72 (1417447057015177217)]} 0 102
[junit4:junit4]   2> 8231 T74 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[28 (1417447056900882432), 29 (1417447056917659648), 30 (1417447056924999680), 33 (1417447056931291136), 35 (1417447056939679744), 37 (1417447056955408384), 39 (1417447056960651264), 40 (1417447056988962816), 41 (1417447057014128641), 42 (1417447057021468672)]} 0 119
[junit4:junit4]   2> 8237 T115 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[86 (1417447056935485441), 87 (1417447056942825472), 88 (1417447056951214080), 89 (1417447056957505536), 90 (1417447056963796993), 92 (1417447056984768512), 93 (1417447056991059968), 97 (1417447057010982912), 102 (1417447057016225793), 103 (1417447057021468673)]} 0 92
[junit4:junit4]   2> 8238 T120 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[95 (1417447056940728321), 96 (1417447056944922625), 98 (1417447056954359808), 100 (1417447056959602688), 101 (1417447056965894144), 102 (1417447056970088448), 106 (1417447056981622785), 107 (1417447056991059968), 111 (1417447057010982912), 116 (1417447057023565825)]} 0 88
[junit4:junit4]   2> 8242 T125 C6 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[106 (1417447056950165504), 108 (1417447056962748416), 114 (1417447056972185600), 115 (1417447056974282752), 117 (1417447056977428480), 120 (1417447056980574208), 121 (1417447057010982912), 123 (1417447057013080064), 125 (1417447057016225792), 127 (1417447057026711553)]} 0 80
[junit4:junit4]   2> 8240 T46 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[25 (1417447056900882432), 29 (1417447056917659648), 30 (1417447056923951104), 32 (1417447056933388288), 33 (1417447056939679744), 38 (1417447056948068353), 40 (1417447056956456961), 42 (1417447056963796992), 43 (1417447056981622784), 47 (1417447057027760128)]} 0 128
[junit4:junit4]   2> 8247 T128 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[117 (1417447056952262656), 119 (1417447056959602689), 121 (1417447056963796993), 122 (1417447056978477056), 125 (1417447056983719936), 126 (1417447056992108545), 128 (1417447057013080064), 129 (1417447057015177216), 131 (1417447057027760129), 132 (1417447057034051584)]} 0 87
[junit4:junit4]   2> 8250 T131 C6 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[128 (1417447056953311232), 130 (1417447056965894145), 131 (1417447056973234176), 133 (1417447056975331328), 135 (1417447056978477056), 136 (1417447056986865665), 137 (1417447056992108544), 138 (1417447057025662976), 139 (1417447057031954433), 144 (1417447057040343040)]} 0 89
[junit4:junit4]   2> 8254 T105 C6 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[56 (1417447056922902528), 58 (1417447056931291136), 59 (1417447056937582592), 60 (1417447056949116928), 67 (1417447056963796992), 68 (1417447056979525633), 69 (1417447056986865664), 76 (1417447057009934336), 78 (1417447057026711552), 79 (1417447057045585920)]} 0 123
[junit4:junit4]   2> 8259 T127 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[109 (1417447056950165504), 110 (1417447056963796992), 111 (1417447056980574208), 115 (1417447056991059968), 117 (1417447057016225792), 118 (1417447057022517248), 119 (1417447057028808704), 122 (1417447057039294465), 127 (1417447057045585920), 128 (1417447057049780224)]} 0 100
[junit4:junit4]   2> 8263 T88 C6 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[33 (1417447056900882432), 34 (1417447056917659648), 38 (1417447056927096832), 40 (1417447056933388288), 41 (1417447056948068352), 43 (1417447056957505536), 46 (1417447056965894144), 48 (1417447056984768513), 50 (1417447057034051584), 55 (1417447057051877377)]} 0 151
[junit4:junit4]   2> 8267 T133 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[129 (1417447056961699841), 131 (1417447056971137024), 132 (1417447056985817088), 135 (1417447057010982912), 137 (1417447057019371522), 138 (1417447057024614401), 142 (1417447057039294464), 143 (1417447057045585921), 144 (1417447057056071680), 147 (1417447057060265985)]} 0 98
[junit4:junit4]   2> 8272 T134 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[122 (1417447056962748416), 124 (1417447056969039872), 125 (1417447056992108546), 126 (1417447057012031488), 131 (1417447057014128642), 132 (1417447057025662976), 134 (1417447057035100160), 135 (1417447057049780224), 152 (1417447057057120257), 154 (1417447057061314560)]} 0 102
[junit4:junit4]   2> 8274 T116 C6 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[95 (1417447056934436864), 96 (1417447056952262656), 97 (1417447056961699841), 98 (1417447056970088448), 99 (1417447057008885760), 100 (1417447057025662977), 101 (1417447057035100161), 102 (1417447057043488768), 104 (1417447057060265984), 105 (1417447057066557440)]} 0 130
[junit4:junit4]   2> 8284 T45 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[184 (1417447057023565824), 187 (1417447057031954432), 188 (1417447057041391616), 192 (1417447057045585922), 195 (1417447057056071680), 197 (1417447057059217408), 198 (1417447057063411712), 202 (1417447057068654592), 203 (1417447057071800321), 204 (1417447057073897472)]} 0 55
[junit4:junit4]   2> 8288 T135 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[134 (1417447056970088449), 137 (1417447056990011393), 141 (1417447057014128640), 143 (1417447057027760130), 146 (1417447057040343040), 147 (1417447057048731648), 149 (1417447057050828800), 150 (1417447057067606017), 151 (1417447057069703169), 152 (1417447057073897473)]} 0 115
[junit4:junit4]   2> 8294 T145 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[198 (1417447057030905856), 202 (1417447057038245888), 203 (1417447057042440192), 204 (1417447057046634496), 205 (1417447057057120256), 207 (1417447057060265985), 210 (1417447057068654592), 211 (1417447057073897472), 212 (1417447057079140352), 217 (1417447057087528962)]} 0 58
[junit4:junit4]   2> 8296 T144 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[156 (1417447057016225792), 157 (1417447057025662977), 158 (1417447057033003008), 161 (1417447057040343040), 162 (1417447057045585920), 164 (1417447057055023104), 165 (1417447057060265984), 167 (1417447057064460288), 170 (1417447057080188929), 172 (1417447057089626112)]} 0 74
[junit4:junit4]   2> 8297 T86 C6 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[197 (1417447057031954432), 198 (1417447057039294465), 199 (1417447057048731648), 203 (1417447057052925952), 205 (1417447057057120256), 207 (1417447057061314560), 214 (1417447057063411713), 216 (1417447057070751745), 217 (1417447057081237504), 219 (1417447057089626113)]} 0 63
[junit4:junit4]   2> 8297 T140 C6 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[145 (1417447057023565824), 146 (1417447057035100160), 148 (1417447057042440192), 149 (1417447057046634497), 153 (1417447057049780224), 155 (1417447057071800320), 158 (1417447057074946049), 159 (1417447057080188928), 161 (1417447057087528961), 170 (1417447057090674689)]} 0 68
[junit4:junit4]   2> 8303 T106 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[218 (1417447057045585921), 220 (1417447057050828800), 221 (1417447057060265986), 223 (1417447057062363136), 225 (1417447057070751744), 226 (1417447057073897473), 227 (1417447057082286081), 229 (1417447057090674689), 230 (1417447057094868994), 231 (1417447057096966145)]} 0 53
[junit4:junit4]   2> 8304 T73 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[173 (1417447057024614400), 174 (1417447057034051584), 176 (1417447057047683072), 178 (1417447057053974528), 179 (1417447057058168832), 185 (1417447057066557440), 186 (1417447057067606016), 189 (1417447057075994624), 193 (1417447057084383232), 195 (1417447057088577536)]} 0 75
[junit4:junit4]   2> 8305 T110 C6 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[220 (1417447057038245888), 224 (1417447057042440193), 228 (1417447057048731650), 229 (1417447057051877378), 232 (1417447057074946048), 234 (1417447057081237505), 236 (1417447057087528960), 237 (1417447057090674688), 238 (1417447057093820416), 239 (1417447057098014720)]} 0 62
[junit4:junit4]   2> 8306 T119 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[186 (1417447057038245888), 189 (1417447057044537344), 190 (1417447057057120257), 192 (1417447057060265984), 193 (1417447057063411712), 196 (1417447057068654593), 197 (1417447057070751745), 201 (1417447057072848896), 205 (1417447057079140352), 206 (1417447057090674688)]} 0 63
[junit4:junit4]   2> 8313 T149 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[278 (1417447057053974528), 279 (1417447057057120260), 280 (1417447057060265986), 281 (1417447057064460288), 282 (1417447057069703169), 283 (1417447057072848897), 284 (1417447057078091776), 286 (1417447057096966144), 287 (1417447057104306176), 290 (1417447057107451905)]} 0 55
[junit4:junit4]   2> 8313 T60 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[207 (1417447057057120258), 209 (1417447057067606016), 210 (1417447057071800320), 211 (1417447057080188928), 215 (1417447057085431808), 216 (1417447057092771840), 217 (1417447057093820418), 219 (1417447057103257601), 224 (1417447057107451904), 225 (1417447057109549056)]} 0 63
[junit4:junit4]   2> 8315 T123 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[301 (1417447057087528960), 304 (1417447057092771840), 306 (1417447057094868992), 308 (1417447057095917568), 312 (1417447057098014721), 313 (1417447057102209024), 315 (1417447057104306177), 321 (1417447057107451904), 322 (1417447057109549056), 323 (1417447057111646208)]} 0 25
[junit4:junit4]   2> 8318 T128 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[228 (1417447057049780224), 229 (1417447057056071681), 230 (1417447057060265984), 231 (1417447057062363136), 233 (1417447057081237504), 234 (1417447057088577537), 238 (1417447057098014720), 239 (1417447057102209024), 240 (1417447057108500480), 241 (1417447057113743360)]} 0 65
[junit4:junit4]   2> 8322 T74 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[325 (1417447057084383233), 326 (1417447057090674688), 327 (1417447057094868993), 330 (1417447057099063296), 332 (1417447057104306176), 334 (1417447057106403328), 335 (1417447057108500480), 336 (1417447057110597632), 338 (1417447057112694784), 339 (1417447057114791937)]} 0 35
[junit4:junit4]   2> 8323 T112 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[300 (1417447057084383233), 302 (1417447057090674688), 303 (1417447057094868992), 306 (1417447057095917569), 313 (1417447057104306176), 316 (1417447057106403329), 319 (1417447057109549056), 320 (1417447057112694785), 321 (1417447057115840513), 322 (1417447057118986241)]} 0 36
[junit4:junit4]   2> 8326 T108 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[151 (1417447057024614400), 152 (1417447057038245889), 154 (1417447057042440192), 155 (1417447057057120261), 160 (1417447057062363136), 162 (1417447057069703168), 163 (1417447057073897472), 164 (1417447057088577536), 165 (1417447057092771841), 172 (1417447057094868993)]} 0 96
[junit4:junit4]   2> 8328 T47 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[262 (1417447057072848896), 267 (1417447057074946048), 269 (1417447057079140353), 273 (1417447057096966144), 274 (1417447057104306177), 275 (1417447057106403328), 276 (1417447057110597633), 277 (1417447057112694784), 279 (1417447057114791936), 281 (1417447057120034816)]} 0 52
[junit4:junit4]   2> 8330 T125 C6 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[242 (1417447057048731649), 245 (1417447057072848897), 249 (1417447057094868992), 250 (1417447057098014721), 251 (1417447057104306176), 255 (1417447057107451904), 256 (1417447057112694784), 265 (1417447057117937665), 267 (1417447057122131969), 269 (1417447057126326272)]} 0 77
[junit4:junit4]   2> 8331 T131 C6 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[270 (1417447057066557441), 272 (1417447057070751744), 275 (1417447057073897472), 276 (1417447057079140352), 277 (1417447057095917568), 284 (1417447057106403328), 286 (1417447057111646208), 287 (1417447057115840512), 293 (1417447057118986241), 296 (1417447057123180544)]} 0 61
[junit4:junit4]   2> 8331 T153 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[337 (1417447057091723264), 340 (1417447057095917568), 341 (1417447057101160448), 342 (1417447057105354752), 343 (1417447057113743360), 344 (1417447057114791936), 347 (1417447057116889089), 348 (1417447057118986240), 349 (1417447057121083392), 351 (1417447057126326272)]} 0 37
[junit4:junit4]   2> 8337 T148 C6 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[174 (1417447057039294464), 176 (1417447057046634496), 178 (1417447057051877376), 186 (1417447057059217408), 187 (1417447057063411712), 188 (1417447057068654592), 189 (1417447057072848896), 192 (1417447057083334656), 195 (1417447057089626112), 196 (1417447057094868993)]} 0 93
[junit4:junit4]   2> 8344 T61 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[173 (1417447057015177216), 174 (1417447057019371520), 175 (1417447057024614402), 176 (1417447057031954432), 177 (1417447057041391616), 179 (1417447057057120256), 181 (1417447057059217408), 182 (1417447057068654592), 183 (1417447057074946048), 184 (1417447057094868992)]} 0 124
[junit4:junit4]   2> 8346 T46 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[282 (1417447057079140352), 283 (1417447057085431809), 284 (1417447057104306178), 285 (1417447057107451904), 286 (1417447057110597632), 289 (1417447057116889088), 294 (1417447057122131968), 296 (1417447057131569152), 297 (1417447057138909185), 299 (1417447057142054912)]} 0 65
[junit4:junit4]   2> 8346 T143 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[207 (1417447057035100160), 213 (1417447057042440192), 215 (1417447057045585920), 216 (1417447057053974528), 218 (1417447057058168832), 221 (1417447057067606016), 222 (1417447057080188928), 223 (1417447057084383232), 224 (1417447057134714880), 225 (1417447057139957760)]} 0 107
[junit4:junit4]   2> 8348 T152 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[226 (1417447057075994624), 230 (1417447057082286080), 233 (1417447057090674689), 234 (1417447057093820416), 237 (1417447057098014720), 238 (1417447057103257600), 244 (1417447057106403328), 246 (1417447057141006336), 248 (1417447057143103489), 250 (1417447057145200640)]} 0 82
[junit4:junit4]   2> 8350 T134 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[345 (1417447057114791936), 346 (1417447057117937664), 347 (1417447057121083392), 351 (1417447057124229120), 354 (1417447057129472000), 355 (1417447057131569153), 357 (1417447057133666304), 359 (1417447057135763456), 360 (1417447057142054913), 361 (1417447057145200640)]} 0 34
[junit4:junit4]   2> 8355 T149 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[388 (1417447057120034817), 390 (1417447057125277696), 392 (1417447057128423425), 393 (1417447057131569152), 396 (1417447057133666305), 397 (1417447057137860608), 398 (1417447057146249216), 399 (1417447057148346368), 403 (1417447057150443521), 404 (1417447057152540672)]} 0 34
[junit4:junit4]   2> 8356 T146 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[254 (1417447057080188928), 255 (1417447057087528961), 256 (1417447057093820417), 257 (1417447057096966144), 258 (1417447057127374848), 261 (1417447057132617728), 264 (1417447057139957760), 266 (1417447057143103488), 267 (1417447057147297792), 268 (1417447057151492096)]} 0 74
[junit4:junit4]   2> 8357 T110 C6 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[298 (1417447057105354752), 299 (1417447057108500480), 303 (1417447057111646209), 305 (1417447057114791936), 306 (1417447057117937664), 307 (1417447057121083392), 308 (1417447057138909184), 314 (1417447057148346368), 315 (1417447057152540674), 317 (1417447057154637827)]} 0 50
[junit4:junit4]   2> 8363 T142 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[154 (1417447057022517248), 155 (1417447057045585921), 156 (1417447057069703168), 158 (1417447057071800320), 160 (1417447057085431808), 161 (1417447057136812032), 162 (1417447057142054913), 163 (1417447057147297792), 169 (1417447057150443521), 175 (1417447057156734977)]} 0 136
[junit4:junit4]   2> 8364 T115 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[269 (1417447057085431808), 271 (1417447057093820416), 273 (1417447057126326272), 275 (1417447057132617729), 276 (1417447057137860609), 279 (1417447057142054912), 283 (1417447057146249216), 291 (1417447057149394944), 294 (1417447057154637824), 300 (1417447057159880706)]} 0 76
[junit4:junit4]   2> 8369 T45 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[348 (1417447057115840512), 349 (1417447057117937664), 351 (1417447057127374849), 354 (1417447057131569153), 355 (1417447057133666304), 357 (1417447057135763457), 358 (1417447057144152064), 359 (1417447057148346371), 362 (1417447057155686400), 364 (1417447057160929282)]} 0 52
[junit4:junit4]   2> 8370 T75 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[232 (1417447057079140353), 233 (1417447057082286080), 235 (1417447057098014720), 236 (1417447057131569152), 237 (1417447057134714880), 238 (1417447057137860608), 239 (1417447057146249217), 243 (1417447057157783552), 246 (1417447057161977857), 248 (1417447057167220736)]} 0 88
[junit4:junit4]   2> 8372 T47 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[406 (1417447057148346370), 409 (1417447057150443522), 410 (1417447057153589248), 412 (1417447057155686401), 415 (1417447057157783552), 416 (1417447057159880704), 417 (1417447057163026432), 418 (1417447057165123584), 420 (1417447057167220736), 421 (1417447057169317888)]} 0 24
[junit4:junit4]   2> 8373 T116 C6 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[368 (1417447057118986240), 369 (1417447057125277697), 370 (1417447057130520576), 372 (1417447057132617728), 373 (1417447057134714881), 374 (1417447057136812032), 377 (1417447057139957761), 382 (1417447057142054912), 383 (1417447057152540672), 384 (1417447057163026432)]} 0 53
[junit4:junit4]   2> 8375 T120 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[244 (1417447057072848897), 248 (1417447057082286080), 250 (1417447057091723264), 251 (1417447057095917568), 252 (1417447057125277696), 253 (1417447057127374850), 256 (1417447057130520576), 257 (1417447057133666305), 259 (1417447057136812033), 261 (1417447057143103488)]} 0 99
[junit4:junit4]   2> 8385 T127 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[293 (1417447057057120259), 294 (1417447057063411713), 295 (1417447057070751744), 297 (1417447057079140353), 298 (1417447057086480384), 299 (1417447057093820417), 302 (1417447057126326273), 304 (1417447057135763457), 308 (1417447057143103488), 312 (1417447057177706496)]} 0 124
[junit4:junit4]   2> 8387 T128 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[365 (1417447057118986240), 367 (1417447057123180544), 368 (1417447057148346369), 370 (1417447057151492096), 371 (1417447057156734976), 375 (1417447057160929281), 377 (1417447057166172160), 379 (1417447057173512192), 380 (1417447057181900801), 382 (1417447057187143680)]} 0 67
[junit4:junit4]   2> 8387 T60 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[370 (1417447057123180544), 372 (1417447057127374848), 373 (1417447057130520576), 377 (1417447057133666304), 379 (1417447057137860609), 380 (1417447057138909184), 384 (1417447057142054912), 385 (1417447057173512192), 386 (1417447057180852224), 387 (1417447057186095106)]} 0 63
[junit4:junit4]   2> 8389 T153 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[444 (1417447057147297792), 445 (1417447057150443520), 446 (1417447057153589248), 447 (1417447057156734976), 449 (1417447057159880704), 451 (1417447057165123584), 452 (1417447057169317888), 453 (1417447057173512194), 454 (1417447057182949376), 457 (1417447057189240832)]} 0 43
[junit4:junit4]   2> 8390 T135 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[323 (1417447057088577536), 328 (1417447057092771840), 329 (1417447057127374848), 331 (1417447057129472001), 332 (1417447057133666306), 336 (1417447057148346368), 337 (1417447057153589249), 343 (1417447057167220737), 344 (1417447057175609344), 345 (1417447057185046529)]} 0 99
[junit4:junit4]   2> 8400 T105 C6 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[425 (1417447057134714880), 426 (1417447057137860609), 429 (1417447057148346369), 432 (1417447057150443520), 433 (1417447057154637824), 434 (1417447057156734976), 435 (1417447057161977856), 436 (1417447057175609344), 438 (1417447057180852225), 439 (1417447057182949377)]} 0 66
[junit4:junit4]   2> 8402 T112 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[385 (1417447057126326272), 386 (1417447057129472000), 387 (1417447057132617728), 389 (1417447057135763456), 392 (1417447057138909184), 393 (1417447057141006336), 394 (1417447057171415040), 396 (1417447057180852225), 399 (1417447057186095104), 401 (1417447057189240832)]} 0 75
[junit4:junit4]   2> 8410 T59 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[252 (1417447057058168832), 256 (1417447057125277697), 259 (1417447057128423424), 261 (1417447057131569153), 263 (1417447057141006337), 264 (1417447057144152064), 265 (1417447057186095104), 266 (1417447057190289408), 271 (1417447057201823744), 272 (1417447057206018048)]} 0 149
[junit4:junit4]   2> 8415 T110 C6 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[477 (1417447057161977857), 480 (1417447057164075012), 481 (1417447057168269313), 482 (1417447057175609345), 484 (1417447057179803648), 486 (1417447057181900800), 487 (1417447057186095104), 488 (1417447057188192256), 489 (1417447057189240832), 493 (1417447057206018048)]} 0 54
[junit4:junit4]   2> 8416 T152 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[484 (1417447057178755072), 487 (1417447057182949377), 489 (1417447057185046528), 493 (1417447057186095108), 494 (1417447057188192256), 497 (1417447057190289411), 499 (1417447057192386560), 500 (1417447057201823745), 503 (1417447057207066624), 505 (1417447057215455232)]} 0 39
[junit4:junit4]   2> 8419 T61 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[458 (1417447057156734977), 459 (1417447057159880705), 462 (1417447057164075008), 465 (1417447057167220736), 467 (1417447057170366464), 468 (1417447057181900800), 474 (1417447057186095107), 476 (1417447057190289409), 477 (1417447057201823746), 479 (1417447057208115202)]} 0 63
[junit4:junit4]   2> 8424 T140 C6 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[319 (1417447057124229120), 321 (1417447057137860610), 322 (1417447057154637826), 329 (1417447057156734977), 332 (1417447057158832128), 334 (1417447057164075010), 335 (1417447057169317889), 336 (1417447057174560768), 339 (1417447057178755073), 340 (1417447057209163776)]} 0 100
[junit4:junit4]   2> 8426 T46 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[442 (1417447057150443520), 444 (1417447057156734978), 448 (1417447057160929280), 449 (1417447057165123585), 451 (1417447057169317889), 453 (1417447057181900800), 455 (1417447057204969472), 456 (1417447057215455234), 457 (1417447057219649536), 459 (1417447057223843840)]} 0 77
[junit4:junit4]   2> 8432 T149 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[423 (1417447057182949378), 424 (1417447057186095105), 425 (1417447057188192257), 427 (1417447057191337984), 428 (1417447057202872320), 429 (1417447057208115201), 430 (1417447057215455233), 431 (1417447057222795264), 439 (1417447057224892417), 443 (1417447057233281024)]} 0 50
[junit4:junit4]   2> 8432 T144 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[464 (1417447057159880705), 468 (1417447057164075008), 469 (1417447057166172160), 470 (1417447057175609344), 472 (1417447057180852224), 479 (1417447057186095104), 480 (1417447057204969472), 481 (1417447057214406656), 482 (1417447057226989568), 484 (1417447057232232448)]} 0 74
[junit4:junit4]   2> 8433 T133 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[313 (1417447057088577537), 314 (1417447057139957760), 317 (1417447057163026432), 321 (1417447057169317889), 322 (1417447057173512193), 325 (1417447057175609344), 328 (1417447057183997952), 329 (1417447057208115200), 332 (1417447057216503809), 336 (1417447057228038144)]} 0 142
[junit4:junit4]   2> 8433 T143 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[474 (1417447057161977856), 475 (1417447057164075008), 476 (1417447057168269312), 478 (1417447057176657920), 479 (1417447057182949377), 480 (1417447057191337984), 485 (1417447057193435137), 487 (1417447057213358080), 488 (1417447057218600960), 490 (1417447057228038144)]} 0 71
[junit4:junit4]   2> 8434 T131 C6 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[441 (1417447057154637825), 442 (1417447057164075008), 445 (1417447057169317888), 446 (1417447057181900801), 453 (1417447057208115201), 454 (1417447057226989568), 455 (1417447057230135296), 457 (1417447057232232448), 458 (1417447057234329600), 459 (1417447057236426752)]} 0 81
[junit4:junit4]   2> 8434 T108 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[407 (1417447057130520577), 408 (1417447057135763456), 409 (1417447057182949379), 410 (1417447057189240833), 412 (1417447057211260928), 414 (1417447057214406656), 415 (1417447057219649536), 417 (1417447057230135296), 419 (1417447057233281025), 420 (1417447057236426752)]} 0 103
[junit4:junit4]   2> 8435 T120 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[511 (1417447057212309504), 515 (1417447057214406656), 519 (1417447057215455235), 521 (1417447057218600961), 524 (1417447057225940992), 525 (1417447057229086721), 527 (1417447057232232448), 528 (1417447057234329600), 529 (1417447057235378177), 531 (1417447057237475329)]} 0 27
[junit4:junit4]   2> 8435 T158 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[422 (1417447057169317890), 424 (1417447057182949376), 426 (1417447057185046528), 427 (1417447057186095105), 433 (1417447057189240833), 434 (1417447057193435136), 435 (1417447057208115200), 438 (1417447057215455233), 440 (1417447057221746689), 441 (1417447057232232450)]} 0 67
[junit4:junit4]   2> 8436 T119 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[356 (1417447057113743361), 357 (1417447057116889088), 358 (1417447057120034816), 359 (1417447057122131968), 361 (1417447057185046529), 363 (1417447057209163776), 364 (1417447057223843841), 365 (1417447057226989568), 367 (1417447057233281026), 369 (1417447057237475328)]} 0 122
[junit4:junit4]   2> 8438 T86 C6 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[386 (1417447057125277696), 394 (1417447057152540673), 396 (1417447057160929280), 397 (1417447057168269312), 398 (1417447057177706497), 399 (1417447057209163778), 401 (1417447057226989569), 402 (1417447057235378176), 403 (1417447057238523904), 404 (1417447057241669632)]} 0 112
[junit4:junit4]   2> 8441 T88 C6 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[341 (1417447057115840513), 343 (1417447057122131968), 345 (1417447057137860608), 348 (1417447057159880704), 350 (1417447057164075011), 352 (1417447057167220736), 356 (1417447057180852224), 359 (1417447057213358080), 361 (1417447057218600960), 367 (1417447057223843840)]} 0 124
[junit4:junit4]   2> 8445 T142 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[492 (1417447057172463616), 496 (1417447057180852224), 497 (1417447057183997952), 498 (1417447057201823744), 499 (1417447057206018048), 502 (1417447057215455232), 504 (1417447057221746688), 506 (1417447057233281024), 509 (1417447057235378176), 510 (1417447057241669632)]} 0 75
[junit4:junit4]   2> 8445 T115 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[416 (1417447057170366464), 417 (1417447057177706496), 425 (1417447057186095105), 430 (1417447057212309506), 432 (1417447057215455233), 433 (1417447057222795264), 437 (1417447057224892416), 438 (1417447057234329600), 439 (1417447057238523904), 442 (1417447057241669633)]} 0 76
[junit4:junit4]   2> 8446 T125 C6 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[407 (1417447057139957760), 408 (1417447057143103488), 409 (1417447057177706498), 411 (1417447057191337984), 412 (1417447057208115200), 416 (1417447057212309505), 417 (1417447057215455232), 418 (1417447057222795264), 421 (1417447057224892416), 423 (1417447057246912512)]} 0 107
[junit4:junit4]   2> 8449 T116 C6 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[494 (1417447057178755072), 495 (1417447057182949376), 496 (1417447057206018049), 500 (1417447057212309504), 501 (1417447057221746688), 502 (1417447057225940992), 503 (1417447057229086721), 505 (1417447057245863938), 510 (1417447057246912513), 511 (1417447057251106816)]} 0 73
[junit4:junit4]   2> 8450 T148 C6 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[460 (1417447057160929281), 463 (1417447057164075009), 464 (1417447057177706496), 465 (1417447057179803649), 467 (1417447057209163777), 469 (1417447057228038144), 472 (1417447057245863936), 473 (1417447057247961088), 474 (1417447057249009664), 475 (1417447057252155393)]} 0 91
[junit4:junit4]   2> 8455 T145 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[503 (1417447057212309504), 504 (1417447057219649537), 506 (1417447057228038145), 508 (1417447057243766785), 510 (1417447057246912512), 512 (1417447057249009665), 513 (1417447057252155393), 515 (1417447057255301120), 517 (1417447057257398272), 518 (1417447057259495424)]} 0 47
[junit4:junit4]   2> 8457 T45 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[532 (1417447057214406657), 537 (1417447057222795265), 539 (1417447057229086720), 540 (1417447057233281025), 541 (1417447057237475328), 544 (1417447057242718208), 546 (1417447057246912512), 547 (1417447057250058240), 548 (1417447057254252544), 549 (1417447057258446848)]} 0 50
[junit4:junit4]   2> 8457 T106 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[485 (1417447057192386560), 488 (1417447057223843840), 489 (1417447057230135296), 491 (1417447057234329601), 492 (1417447057237475328), 493 (1417447057240621056), 494 (1417447057243766786), 495 (1417447057246912513), 500 (1417447057253203968), 501 (1417447057256349696)]} 0 68
[junit4:junit4]   2> 8461 T162 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[551 (1417447057223843840), 552 (1417447057228038145), 553 (1417447057232232448), 556 (1417447057235378176), 560 (1417447057238523904), 561 (1417447057240621057), 562 (1417447057243766784), 564 (1417447057246912513), 565 (1417447057250058240), 566 (1417447057261592576)]} 0 42
[junit4:junit4]   2> 8461 T73 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&vers

[...truncated too long message...]

   >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   11) Thread[id=26, name=qtp1484531491-26 Acceptor0 SelectChannelConnector@0.0.0.0:26902, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   12) Thread[id=56, name=qtp907466781-56 Acceptor2 SelectChannelConnector@0.0.0.0:33240, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   13) Thread[id=29, name=qtp1484531491-29 Acceptor3 SelectChannelConnector@0.0.0.0:26902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   14) Thread[id=27, name=qtp1484531491-27 Acceptor1 SelectChannelConnector@0.0.0.0:26902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   15) Thread[id=42, name=qtp593385024-42 Acceptor2 SelectChannelConnector@0.0.0.0:38543, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   16) Thread[id=70, name=qtp1364030911-70 Acceptor2 SelectChannelConnector@0.0.0.0:59424, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   17) Thread[id=57, name=qtp907466781-57 Acceptor3 SelectChannelConnector@0.0.0.0:33240, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   18) Thread[id=68, name=qtp1364030911-68 Acceptor0 SelectChannelConnector@0.0.0.0:59424, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   19) Thread[id=84, name=qtp326170431-84 Acceptor2 SelectChannelConnector@0.0.0.0:59797, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   20) Thread[id=55, name=qtp907466781-55 Acceptor1 SelectChannelConnector@0.0.0.0:33240, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([8ADC3589EAB97A9]:0)
[junit4:junit4] Completed on J0 in 34.49s, 1 test, 2 errors <<< FAILURES!

[...truncated 518 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:335: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/module-build.xml:61: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:1163: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:827: There were test failures: 233 suites, 965 tests, 4 suite-level errors, 822 ignored

Total time: 18 minutes 3 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure

[JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 603 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.

Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/603/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
8 threads leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:     1) Thread[id=407, name=qtp680254087-407 Acceptor3 SelectChannelConnector@0.0.0.0:22157, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=406, name=qtp680254087-406 Acceptor2 SelectChannelConnector@0.0.0.0:22157, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=405, name=qtp680254087-405 Acceptor1 SelectChannelConnector@0.0.0.0:22157, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=338, name=qtp551926596-338 Acceptor2 SelectChannelConnector@0.0.0.0:57705, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=336, name=qtp551926596-336 Acceptor0 SelectChannelConnector@0.0.0.0:57705, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=337, name=qtp551926596-337 Acceptor1 SelectChannelConnector@0.0.0.0:57705, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=339, name=qtp551926596-339 Acceptor3 SelectChannelConnector@0.0.0.0:57705, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=404, name=qtp680254087-404 Acceptor0 SelectChannelConnector@0.0.0.0:22157, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 8 threads leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest: 
   1) Thread[id=407, name=qtp680254087-407 Acceptor3 SelectChannelConnector@0.0.0.0:22157, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=406, name=qtp680254087-406 Acceptor2 SelectChannelConnector@0.0.0.0:22157, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=405, name=qtp680254087-405 Acceptor1 SelectChannelConnector@0.0.0.0:22157, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=338, name=qtp551926596-338 Acceptor2 SelectChannelConnector@0.0.0.0:57705, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=336, name=qtp551926596-336 Acceptor0 SelectChannelConnector@0.0.0.0:57705, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=337, name=qtp551926596-337 Acceptor1 SelectChannelConnector@0.0.0.0:57705, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=339, name=qtp551926596-339 Acceptor3 SelectChannelConnector@0.0.0.0:57705, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=404, name=qtp680254087-404 Acceptor0 SelectChannelConnector@0.0.0.0:22157, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([10754BD6AFFFB7CB]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=407, name=qtp680254087-407 Acceptor3 SelectChannelConnector@0.0.0.0:22157, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=406, name=qtp680254087-406 Acceptor2 SelectChannelConnector@0.0.0.0:22157, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=405, name=qtp680254087-405 Acceptor1 SelectChannelConnector@0.0.0.0:22157, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=338, name=qtp551926596-338 Acceptor2 SelectChannelConnector@0.0.0.0:57705, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=336, name=qtp551926596-336 Acceptor0 SelectChannelConnector@0.0.0.0:57705, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=337, name=qtp551926596-337 Acceptor1 SelectChannelConnector@0.0.0.0:57705, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=339, name=qtp551926596-339 Acceptor3 SelectChannelConnector@0.0.0.0:57705, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=404, name=qtp680254087-404 Acceptor0 SelectChannelConnector@0.0.0.0:22157, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=407, name=qtp680254087-407 Acceptor3 SelectChannelConnector@0.0.0.0:22157, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=406, name=qtp680254087-406 Acceptor2 SelectChannelConnector@0.0.0.0:22157, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=405, name=qtp680254087-405 Acceptor1 SelectChannelConnector@0.0.0.0:22157, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=338, name=qtp551926596-338 Acceptor2 SelectChannelConnector@0.0.0.0:57705, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=336, name=qtp551926596-336 Acceptor0 SelectChannelConnector@0.0.0.0:57705, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=337, name=qtp551926596-337 Acceptor1 SelectChannelConnector@0.0.0.0:57705, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=339, name=qtp551926596-339 Acceptor3 SelectChannelConnector@0.0.0.0:57705, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=404, name=qtp680254087-404 Acceptor0 SelectChannelConnector@0.0.0.0:22157, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([10754BD6AFFFB7CB]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.RecoveryZkTest

Error Message:
12 threads leaked from SUITE scope at org.apache.solr.cloud.RecoveryZkTest:     1) Thread[id=207, name=qtp930016431-207 Acceptor0 SelectChannelConnector@0.0.0.0:21225, state=RUNNABLE, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=210, name=qtp930016431-210 Acceptor3 SelectChannelConnector@0.0.0.0:21225, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=300, name=qtp1564709036-300 Acceptor0 SelectChannelConnector@0.0.0.0:21235, state=RUNNABLE, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=227, name=qtp50250627-227 Acceptor2 SelectChannelConnector@0.0.0.0:21230, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=226, name=qtp50250627-226 Acceptor1 SelectChannelConnector@0.0.0.0:21230, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=208, name=qtp930016431-208 Acceptor1 SelectChannelConnector@0.0.0.0:21225, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=225, name=qtp50250627-225 Acceptor0 SelectChannelConnector@0.0.0.0:21230, state=RUNNABLE, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=301, name=qtp1564709036-301 Acceptor1 SelectChannelConnector@0.0.0.0:21235, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=209, name=qtp930016431-209 Acceptor2 SelectChannelConnector@0.0.0.0:21225, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=228, name=qtp50250627-228 Acceptor3 SelectChannelConnector@0.0.0.0:21230, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=302, name=qtp1564709036-302 Acceptor2 SelectChannelConnector@0.0.0.0:21235, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=303, name=qtp1564709036-303 Acceptor3 SelectChannelConnector@0.0.0.0:21235, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 12 threads leaked from SUITE scope at org.apache.solr.cloud.RecoveryZkTest: 
   1) Thread[id=207, name=qtp930016431-207 Acceptor0 SelectChannelConnector@0.0.0.0:21225, state=RUNNABLE, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=210, name=qtp930016431-210 Acceptor3 SelectChannelConnector@0.0.0.0:21225, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=300, name=qtp1564709036-300 Acceptor0 SelectChannelConnector@0.0.0.0:21235, state=RUNNABLE, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=227, name=qtp50250627-227 Acceptor2 SelectChannelConnector@0.0.0.0:21230, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=226, name=qtp50250627-226 Acceptor1 SelectChannelConnector@0.0.0.0:21230, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=208, name=qtp930016431-208 Acceptor1 SelectChannelConnector@0.0.0.0:21225, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=225, name=qtp50250627-225 Acceptor0 SelectChannelConnector@0.0.0.0:21230, state=RUNNABLE, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=301, name=qtp1564709036-301 Acceptor1 SelectChannelConnector@0.0.0.0:21235, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=209, name=qtp930016431-209 Acceptor2 SelectChannelConnector@0.0.0.0:21225, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=228, name=qtp50250627-228 Acceptor3 SelectChannelConnector@0.0.0.0:21230, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=302, name=qtp1564709036-302 Acceptor2 SelectChannelConnector@0.0.0.0:21235, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=303, name=qtp1564709036-303 Acceptor3 SelectChannelConnector@0.0.0.0:21235, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([10754BD6AFFFB7CB]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.RecoveryZkTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=301, name=qtp1564709036-301 Acceptor1 SelectChannelConnector@0.0.0.0:21235, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=207, name=qtp930016431-207 Acceptor0 SelectChannelConnector@0.0.0.0:21225, state=RUNNABLE, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=210, name=qtp930016431-210 Acceptor3 SelectChannelConnector@0.0.0.0:21225, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=300, name=qtp1564709036-300 Acceptor0 SelectChannelConnector@0.0.0.0:21235, state=RUNNABLE, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=227, name=qtp50250627-227 Acceptor2 SelectChannelConnector@0.0.0.0:21230, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=226, name=qtp50250627-226 Acceptor1 SelectChannelConnector@0.0.0.0:21230, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=209, name=qtp930016431-209 Acceptor2 SelectChannelConnector@0.0.0.0:21225, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=228, name=qtp50250627-228 Acceptor3 SelectChannelConnector@0.0.0.0:21230, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=302, name=qtp1564709036-302 Acceptor2 SelectChannelConnector@0.0.0.0:21235, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=303, name=qtp1564709036-303 Acceptor3 SelectChannelConnector@0.0.0.0:21235, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=208, name=qtp930016431-208 Acceptor1 SelectChannelConnector@0.0.0.0:21225, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=225, name=qtp50250627-225 Acceptor0 SelectChannelConnector@0.0.0.0:21230, state=RUNNABLE, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=301, name=qtp1564709036-301 Acceptor1 SelectChannelConnector@0.0.0.0:21235, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=207, name=qtp930016431-207 Acceptor0 SelectChannelConnector@0.0.0.0:21225, state=RUNNABLE, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=210, name=qtp930016431-210 Acceptor3 SelectChannelConnector@0.0.0.0:21225, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=300, name=qtp1564709036-300 Acceptor0 SelectChannelConnector@0.0.0.0:21235, state=RUNNABLE, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=227, name=qtp50250627-227 Acceptor2 SelectChannelConnector@0.0.0.0:21230, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=226, name=qtp50250627-226 Acceptor1 SelectChannelConnector@0.0.0.0:21230, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=209, name=qtp930016431-209 Acceptor2 SelectChannelConnector@0.0.0.0:21225, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=228, name=qtp50250627-228 Acceptor3 SelectChannelConnector@0.0.0.0:21230, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=302, name=qtp1564709036-302 Acceptor2 SelectChannelConnector@0.0.0.0:21235, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=303, name=qtp1564709036-303 Acceptor3 SelectChannelConnector@0.0.0.0:21235, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=208, name=qtp930016431-208 Acceptor1 SelectChannelConnector@0.0.0.0:21225, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=225, name=qtp50250627-225 Acceptor0 SelectChannelConnector@0.0.0.0:21230, state=RUNNABLE, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([10754BD6AFFFB7CB]:0)




Build Log:
[...truncated 8368 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.RecoveryZkTest
[junit4:junit4]   2> 5 T191 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-RecoveryZkTest-1351763754330
[junit4:junit4]   2> 6 T191 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 10 T192 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 11 T192 oazs.NIOServerCnxn$Factory.<init> binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 12 T192 oazsp.FileTxnSnapLog.save Snapshotting: 0
[junit4:junit4]   2> 107 T191 oasc.ZkTestServer.run start zk server on port:21221
[junit4:junit4]   2> 107 T191 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:21221 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@16432f65
[junit4:junit4]   2> 108 T197 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:21221
[junit4:junit4]   2> 108 T191 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 109 T197 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:21221, initiating session
[junit4:junit4]   2> 110 T193 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:21222
[junit4:junit4]   2> 110 T193 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:21222
[junit4:junit4]   2> 111 T195 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 150 T195 oazs.NIOServerCnxn.finishSessionInit Established session 0x13abb65fd610000 with negotiated timeout 10000 for client /140.211.11.196:21222
[junit4:junit4]   2> 150 T197 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:21221, sessionid = 0x13abb65fd610000, negotiated timeout = 10000
[junit4:junit4]   2> 151 T198 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16432f65 name:ZooKeeperConnection Watcher:127.0.0.1:21221 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 151 T191 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 152 T191 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 155 T196 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x13abb65fd610000
[junit4:junit4]   2> 157 T198 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 157 T191 oaz.ZooKeeper.close Session: 0x13abb65fd610000 closed
[junit4:junit4]   2> 157 T193 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:21222 which had sessionid 0x13abb65fd610000
[junit4:junit4]   2> 158 T191 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:21221/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@10fc040
[junit4:junit4]   2> 159 T199 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:21221
[junit4:junit4]   2> 159 T191 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 160 T199 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:21221, initiating session
[junit4:junit4]   2> 160 T193 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:21223
[junit4:junit4]   2> 161 T193 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:21223
[junit4:junit4]   2> 162 T195 oazs.NIOServerCnxn.finishSessionInit Established session 0x13abb65fd610001 with negotiated timeout 10000 for client /140.211.11.196:21223
[junit4:junit4]   2> 162 T199 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:21221, sessionid = 0x13abb65fd610001, negotiated timeout = 10000
[junit4:junit4]   2> 163 T200 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10fc040 name:ZooKeeperConnection Watcher:127.0.0.1:21221/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 163 T191 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 164 T191 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 168 T191 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 172 T191 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 183 T191 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 187 T191 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 208 T191 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 222 T191 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 223 T191 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 327 T191 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 328 T191 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 339 T191 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 340 T191 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 350 T191 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 351 T191 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 423 T191 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 424 T191 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 429 T191 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 430 T191 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 478 T191 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 480 T191 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 484 T191 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 485 T191 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 526 T191 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 527 T191 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 551 T196 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x13abb65fd610001
[junit4:junit4]   2> 592 T200 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 592 T191 oaz.ZooKeeper.close Session: 0x13abb65fd610001 closed
[junit4:junit4]   2> 592 T193 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:21223 which had sessionid 0x13abb65fd610001
[junit4:junit4]   2> 1160 T191 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 1435 T191 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:21225
[junit4:junit4]   2> 1464 T191 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1464 T191 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 1465 T191 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 1500 T191 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1500 T191 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1501 T191 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 1501 T191 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 1501 T191 oasc.CoreContainer.<init> New CoreContainer 1184377691
[junit4:junit4]   2> 1502 T191 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 1503 T191 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 1540 T191 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 1555 T191 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:21221/solr
[junit4:junit4]   2> 1572 T191 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:21221/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@1e60a6ce
[junit4:junit4]   2> 1573 T211 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:21221
[junit4:junit4]   2> 1573 T191 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1575 T211 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:21221, initiating session
[junit4:junit4]   2> 1576 T193 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:21226
[junit4:junit4]   2> 1576 T193 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:21226
[junit4:junit4]   2> 1578 T195 oazs.NIOServerCnxn.finishSessionInit Established session 0x13abb65fd610002 with negotiated timeout 5000 for client /140.211.11.196:21226
[junit4:junit4]   2> 1578 T211 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:21221, sessionid = 0x13abb65fd610002, negotiated timeout = 5000
[junit4:junit4]   2> 1579 T212 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e60a6ce name:ZooKeeperConnection Watcher:127.0.0.1:21221/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1579 T191 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1582 T196 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13abb65fd610002 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1596 T191 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 1598 T191 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:21225_solr
[junit4:junit4]   2> 1599 T196 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13abb65fd610002 type:delete cxid:0x7 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:21225_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:21225_solr
[junit4:junit4]   2> 1604 T191 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:21225_solr
[junit4:junit4]   2> 1609 T191 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 1609 T191 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 1610 T191 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 1610 T191 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 1610 T191 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 1611 T191 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 1611 T191 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 1612 T191 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 1612 T191 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 1613 T191 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 1625 T191 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 1634 T196 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13abb65fd610002 type:delete cxid:0x15 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 1643 T191 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 1647 T191 oasc.Overseer.start Overseer (id=88589189404229634-127.0.0.1:21225_solr-n_0000000000) starting
[junit4:junit4]   2> 1647 T196 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13abb65fd610002 type:create cxid:0x1a zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1655 T196 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13abb65fd610002 type:create cxid:0x1b zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1657 T196 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13abb65fd610002 type:create cxid:0x1c zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1661 T196 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13abb65fd610002 type:create cxid:0x1d zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1668 T214 oasc.OverseerCollectionProcessor.run Process current queue of collection messages
[junit4:junit4]   2> 1668 T191 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 1671 T191 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1687 T213 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 1716 T191 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 1717 T191 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 1718 T191 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1718 T191 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 1720 T191 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 1722 T191 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 1722 T191 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 1835 T191 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 1898 T191 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 2000 T191 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 2042 T191 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 2808 T191 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 2829 T191 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 2834 T191 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 2869 T191 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2877 T191 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2885 T191 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2890 T191 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2896 T191 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1351763754329/control/data/
[junit4:junit4]   2> 2902 T191 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e2f10c2
[junit4:junit4]   2> 2944 T191 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1351763754329/control/data/index/
[junit4:junit4]   2> 2946 T191 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1351763754329/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2947 T191 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-1351763754329/control/data/index forceNew:false
[junit4:junit4]   2> 2959 T191 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1351763754329/control/data/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 2960 T191 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 2962 T191 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 2966 T191 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2966 T191 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 2967 T191 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2968 T191 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 2970 T191 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2970 T191 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2971 T191 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2973 T191 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2974 T191 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2975 T191 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 2977 T191 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 2978 T191 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2979 T191 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 2980 T191 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 2981 T191 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 2982 T191 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2983 T191 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2985 T191 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2986 T191 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2987 T191 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2989 T191 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2990 T191 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2992 T191 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2993 T191 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 2995 T191 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 3011 T191 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 3017 T191 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 3019 T191 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 3026 T191 oass.SolrIndexSearcher.<init> Opening Searcher@4cde17e5 main
[junit4:junit4]   2> 3056 T191 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 3057 T191 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 3058 T191 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 3068 T191 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 3125 T219 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4cde17e5 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 3140 T191 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3141 T196 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13abb65fd610002 type:create cxid:0x56 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 3153 T191 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 3154 T191 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:21225/solr shardId:control_shard
[junit4:junit4]   2> 3170 T191 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 3192 T191 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/control_shard/election
[junit4:junit4]   2> 3204 T213 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3206 T213 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"control_shard",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:21225_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:21225/solr"}
[junit4:junit4]   2> 3208 T196 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13abb65fd610002 type:delete cxid:0x6a zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 3217 T191 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 3218 T196 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13abb65fd610002 type:create cxid:0x6b zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3222 T196 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13abb65fd610002 type:create cxid:0x6d zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 3242 T212 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (1)
[junit4:junit4]   2> 3723 T191 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 3724 T191 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 3724 T191 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:21225/solr/collection1/
[junit4:junit4]   2> 3725 T191 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 3726 T191 oasc.SyncStrategy.syncToMe http://127.0.0.1:21225/solr/collection1/ has no replicas
[junit4:junit4]   2> 3727 T191 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:21225/solr/collection1/
[junit4:junit4]   2> 3727 T191 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/control_shard
[junit4:junit4]   2> 3734 T196 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13abb65fd610002 type:create cxid:0x8b zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3750 T213 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3762 T212 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (1)
[junit4:junit4]   2> 3796 T191 oasc.ZkController.register We are http://127.0.0.1:21225/solr/collection1/ and leader is http://127.0.0.1:21225/solr/collection1/
[junit4:junit4]   2> 3800 T191 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:21225/solr
[junit4:junit4]   2> 3801 T191 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 3802 T191 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3804 T191 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3806 T191 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 3807 T191 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3879 T191 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 4109 T191 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 4113 T191 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:21230
[junit4:junit4]   2> 4114 T191 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4115 T191 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351763758228
[junit4:junit4]   2> 4115 T191 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351763758228/'
[junit4:junit4]   2> 4168 T191 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 4169 T191 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4169 T191 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351763758228
[junit4:junit4]   2> 4170 T191 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351763758228/solr.xml
[junit4:junit4]   2> 4171 T191 oasc.CoreContainer.<init> New CoreContainer 1232784122
[junit4:junit4]   2> 4172 T191 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351763758228/'
[junit4:junit4]   2> 4172 T191 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351763758228/'
[junit4:junit4]   2> 4214 T191 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4235 T191 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:21221/solr
[junit4:junit4]   2> 4235 T191 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:21221/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@61c815f9
[junit4:junit4]   2> 4237 T229 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:21221
[junit4:junit4]   2> 4237 T191 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4238 T229 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:21221, initiating session
[junit4:junit4]   2> 4239 T193 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:21231
[junit4:junit4]   2> 4239 T193 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:21231
[junit4:junit4]   2> 4242 T195 oazs.NIOServerCnxn.finishSessionInit Established session 0x13abb65fd610003 with negotiated timeout 5000 for client /140.211.11.196:21231
[junit4:junit4]   2> 4242 T229 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:21221, sessionid = 0x13abb65fd610003, negotiated timeout = 5000
[junit4:junit4]   2> 4243 T230 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@61c815f9 name:ZooKeeperConnection Watcher:127.0.0.1:21221/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4244 T191 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4245 T196 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13abb65fd610003 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4247 T196 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13abb65fd610003 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4253 T191 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:21230_solr
[junit4:junit4]   2> 4255 T196 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13abb65fd610003 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:21230_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:21230_solr
[junit4:junit4]   2> 4256 T191 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:21230_solr
[junit4:junit4]   2> 4265 T212 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4266 T191 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 4267 T191 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4267 T191 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 4268 T213 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4268 T191 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4269 T191 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4270 T213 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"control_shard",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:21225_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:21225/solr"}
[junit4:junit4]   2> 4269 T191 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4270 T191 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4271 T191 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4272 T191 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4273 T191 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 4284 T212 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 4297 T191 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4307 T191 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351763758228/collection1
[junit4:junit4]   2> 4308 T191 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 4309 T191 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 4310 T191 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 4312 T191 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351763758228/collection1/'
[junit4:junit4]   2> 4314 T191 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-jetty1-1351763758228/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4315 T191 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-jetty1-1351763758228/collection1/lib/README' to classloader
[junit4:junit4]   2> 4473 T191 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 4593 T191 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 4695 T191 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4747 T191 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 5660 T191 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 5691 T191 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 5708 T191 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5752 T191 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5763 T191 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5771 T191 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5775 T191 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5779 T191 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351763758228/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1351763754329/jetty1/
[junit4:junit4]   2> 5780 T191 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e2f10c2
[junit4:junit4]   2> 5782 T191 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1351763754329/jetty1/index/
[junit4:junit4]   2> 5783 T191 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1351763754329/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5784 T191 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-1351763754329/jetty1/index forceNew:false
[junit4:junit4]   2> 5792 T191 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1351763754329/jetty1/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5793 T191 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 5794 T191 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 5799 T191 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 5799 T191 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 5801 T191 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 5802 T191 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 5803 T191 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 5804 T191 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 5805 T191 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 5806 T191 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 5808 T191 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 5809 T191 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 5811 T191 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 5812 T191 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 5813 T191 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 5814 T191 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 5815 T191 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 5816 T191 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 5818 T191 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5819 T191 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5821 T191 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5822 T191 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5823 T191 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5825 T191 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5826 T191 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5828 T191 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 5829 T191 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 5843 T191 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 5849 T191 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 5851 T191 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 5858 T191 oass.SolrIndexSearcher.<init> Opening Searcher@310a7aa1 main
[junit4:junit4]   2> 5859 T191 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 5860 T191 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 5862 T191 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 5871 T191 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 5944 T235 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@310a7aa1 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5958 T191 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6299 T213 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6300 T213 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:21230_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:21230/solr"}
[junit4:junit4]   2> 6301 T213 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4]   2> 6318 T212 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 6318 T230 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 6462 T191 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 6463 T191 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:21230/solr shardId:shard1
[junit4:junit4]   2> 6464 T191 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 6482 T196 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13abb65fd610003 type:delete cxid:0x42 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 6483 T191 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 6484 T196 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13abb65fd610003 type:create cxid:0x43 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6493 T191 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 6494 T191 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 6495 T191 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:21230/solr/collection1/
[junit4:junit4]   2> 6495 T191 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 6496 T191 oasc.SyncStrategy.syncToMe http://127.0.0.1:21230/solr/collection1/ has no replicas
[junit4:junit4]   2> 6497 T191 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:21230/solr/collection1/
[junit4:junit4]   2> 6497 T191 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 6503 T196 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13abb65fd610003 type:create cxid:0x4d zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6837 T213 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6864 T212 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 6864 T230 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 6870 T191 oasc.ZkController.register We are http://127.0.0.1:21230/solr/collection1/ and leader is http://127.0.0.1:21230/solr/collection1/
[junit4:junit4]   2> 6870 T191 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:21230/solr
[junit4:junit4]   2> 6871 T191 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 6872 T191 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6876 T191 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6878 T191 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 6878 T191 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 6879 T191 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 7092 T191 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 7096 T191 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:21235
[junit4:junit4]   2> 7097 T191 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 7097 T191 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty2-1351763761222
[junit4:junit4]   2> 7098 T191 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty2-1351763761222/'
[junit4:junit4]   2> 7149 T191 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 7150 T191 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 7151 T191 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty2-1351763761222
[junit4:junit4]   2> 7151 T191 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty2-1351763761222/solr.xml
[junit4:junit4]   2> 7152 T191 oasc.CoreContainer.<init> New CoreContainer 1000606924
[junit4:junit4]   2> 7153 T191 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty2-1351763761222/'
[junit4:junit4]   2> 7154 T191 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty2-1351763761222/'
[junit4:junit4]   2> 7193 T191 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 7214 T191 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:21221/solr
[junit4:junit4]   2> 7215 T191 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:21221/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@c32ddc5
[junit4:junit4]   2> 7216 T245 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:21221
[junit4:junit4]   2> 7216 T191 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7217 T245 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:21221, initiating session
[junit4:junit4]   2> 7217 T193 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:21237
[junit4:junit4]   2> 7218 T193 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:21237
[junit4:junit4]   2> 7221 T195 oazs.NIOServerCnxn.finishSessionInit Established session 0x13abb65fd610004 with negotiated timeout 5000 for client /140.211.11.196:21237
[junit4:junit4]   2> 7221 T245 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:21221, sessionid = 0x13abb65fd610004, negotiated timeout = 5000
[junit4:junit4]   2> 7222 T246 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c32ddc5 name:ZooKeeperConnection Watcher:127.0.0.1:21221/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7222 T191 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7223 T196 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13abb65fd610004 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7225 T196 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13abb65fd610004 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7236 T191 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:21235_solr
[junit4:junit4]   2> 7237 T196 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13abb65fd610004 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:21235_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:21235_solr
[junit4:junit4]   2> 7239 T191 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:21235_solr
[junit4:junit4]   2> 7243 T212 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7243 T230 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7244 T191 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 7244 T191 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 7245 T191 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 7246 T191 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 7246 T191 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 7247 T191 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 7247 T191 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 7248 T191 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 7249 T191 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 7249 T191 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 7273 T191 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 7282 T191 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty2-1351763761222/collection1
[junit4:junit4]   2> 7282 T191 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 7284 T191 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 7284 T191 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 7286 T191 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty2-1351763761222/collection1/'
[junit4:junit4]   2> 7288 T191 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-jetty2-1351763761222/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 7289 T191 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-jetty2-1351763761222/collection1/lib/README' to classloader
[junit4:junit4]   2> 7372 T191 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 7378 T213 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 7379 T213 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:21230_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:21230/solr"}
[junit4:junit4]   2> 7393 T212 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 7393 T230 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 7393 T246 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 7464 T191 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 7565 T191 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 7604 T191 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 8385 T191 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 8399 T191 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 8402 T191 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 8424 T191 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8430 T191 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8435 T191 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8438 T191 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8441 T191 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty2-1351763761222/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1351763754329/jetty2/
[junit4:junit4]   2> 8441 T191 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e2f10c2
[junit4:junit4]   2> 8443 T191 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1351763754329/jetty2/index/
[junit4:junit4]   2> 8443 T191 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1351763754329/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 8444 T191 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-1351763754329/jetty2/index forceNew:false
[junit4:junit4]   2> 8450 T191 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1351763754329/jetty2/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 8451 T191 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 8452 T191 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 8456 T191 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 8457 T191 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 8458 T191 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 8458 T191 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 8460 T191 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 8461 T191 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 8462 T191 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 8463 T191 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 8465 T191 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 8466 T191 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 8467 T191 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 8468 T191 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 8470 T191 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 8470 T191 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 8471 T191 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 8472 T191 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 8474 T191 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8475 T191 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8476 T191 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8478 T191 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8479 T191 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8480 T191 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8481 T191 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8482 T191 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 8483 T191 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 8492 T191 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 8496 T191 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 8498 T191 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 8501 T191 oass.SolrIndexSearcher.<init> Opening Searcher@72132192 main
[junit4:junit4]   2> 8502 T191 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 8503 T191 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 8504 T191 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 8512 T191 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 8561 T250 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@72132192 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 8570 T191 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8905 T213 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8906 T213 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:21235_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:21235/solr"}
[junit4:junit4]   2> 8925 T212 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 8925 T246 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 8925 T230 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 9073 T191 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 9073 T191 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:21235/solr shardId:shard1
[junit4:junit4]   2> 9077 T191 oasc.ZkController.register We are http://127.0.0.1:21235/solr/collection1/ and leader is http://127.0.0.1:21230/solr/collection1/
[junit4:junit4]   2> 9078 T191 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:21235/solr
[junit4:junit4]   2> 9078 T191 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 9079 T191 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C2 name=collection1 org.apache.solr.core.SolrCore@28e66cff url=http://127.0.0.1:21235/solr/collection1 node=127.0.0.1:21235_solr
[junit4:junit4]   2> 9085 T251 C2 P21235 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 9086 T251 C2 P21235 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 9086 T191 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9087 T251 C2 P21235 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9087 T191 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 9088 T191 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 9089 T191 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9097 T251 C2 P21235 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:21230/solr/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 9108 T251 C2 P21235 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 9112 T191 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:21221/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@487c05c8
[junit4:junit4]   2> 9113 T252 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:21221
[junit4:junit4]   2> 9113 T191 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 9114 T252 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:21221, initiating session
[junit4:junit4]   2> 9114 T193 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:21240
[junit4:junit4]   2> 9115 T193 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:21240
[junit4:junit4]   2> 9117 T195 oazs.NIOServerCnxn.finishSessionInit Established session 0x13abb65fd610005 with negotiated timeout 10000 for client /140.211.11.196:21240
[junit4:junit4]   2> 9117 T252 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:21221, sessionid = 0x13abb65fd610005, negotiated timeout = 10000
[junit4:junit4]   2> 9117 T253 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@487c05c8 name:ZooKeeperConnection Watcher:127.0.0.1:21221/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 9117 T191 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 9118 T191 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 9123 T191 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 9124 T191 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 9130 T251 C2 P21235 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:21235/solr START replicas=[http://127.0.0.1:21230/solr/collection1/] nUpdates=100
[junit4:junit4]   2> 9139 T191 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9150 T251 C2 P21235 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 9150 T251 C2 P21235 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 9151 T251 C2 P21235 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 9151 T251 C2 P21235 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9431 T213 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9432 T213 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:21235_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:21235/solr"}
[junit4:junit4]   2> 9446 T212 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 9446 T230 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 9446 T253 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 9446 T246 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> ASYNC  NEW_CORE C3 name=collection1 org.apache.solr.core.SolrCore@79b8e347 url=http://127.0.0.1:21230/solr/collection1 node=127.0.0.1:21230_solr
[junit4:junit4]   2> 10204 T233 C3 P21230 REQ /get {distrib=false&qt=/get&wt=javabin&version=2&getVersions=100} status=0 QTime=20 
[junit4:junit4]   2> ASYNC  NEW_CORE C4 name=collection1 org.apache.solr.core.SolrCore@745fc59f url=http://127.0.0.1:21225/solr/collection1 node=127.0.0.1:21225_solr
[junit4:junit4]   2> 10263 T218 C4 P21225 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16
[junit4:junit4]   2> 10334 T216 C4 P21225 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1351763754329/control/data/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 10335 T216 C4 P21225 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 10380 T216 C4 P21225 /update {wt=javabin&version=2} {add=[10001 (1417427041282686976)]} 0 136
[junit4:junit4]   2> 10380 T218 C4 P21225 /update {wt=javabin&version=2} {add=[1 (1417427041282686977)]} 0 136
[junit4:junit4]   2> 10386 T255 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:21221/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@717ec56c
[junit4:junit4]   2> 10388 T257 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:21221
[junit4:junit4]   2> 10388 T255 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 10389 T257 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:21221, initiating session
[junit4:junit4]   2> 10389 T193 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:21247
[junit4:junit4]   2> 10390 T193 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:21247
[junit4:junit4]   2> 10445 T195 oazs.NIOServerCnxn.finishSessionInit Established session 0x13abb65fd610006 with negotiated timeout 10000 for client /140.211.11.196:21247
[junit4:junit4]   2> 10445 T257 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:21221, sessionid = 0x13abb65fd610006, negotiated timeout = 10000
[junit4:junit4]   2> 10446 T258 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@717ec56c name:ZooKeeperConnection Watcher:127.0.0.1:21221/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 10446 T255 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 10448 T255 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 10481 T234 C3 P21230 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1351763754329/jetty1/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 10482 T234 C3 P21230 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2>  C2_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:21235_solr, base_url=http://127.0.0.1:21235/solr}
[junit4:junit4]   2> 10552 T249 C2 P21235 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1351763754329/jetty2/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 10553 T249 C2 P21235 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 10561 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=15 
[junit4:junit4]   2> 10563 T234 C3 P21230 /update {wt=javabin&version=2} {add=[1 (1417427041438924801)]} 0 96
[junit4:junit4]   2> 10565 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=19 
[junit4:junit4]   2> 10566 T233 C3 P21230 /update {wt=javabin&version=2} {add=[10001 (1417427041438924800)]} 0 100
[junit4:junit4]   2> 10575 T217 C4 P21225 /update {wt=javabin&version=2} {add=[2 (1417427041537490944)]} 0 6
[junit4:junit4]   2> 10578 T216 C4 P21225 /update {wt=javabin&version=2} {add=[10002 (1417427041539588096)]} 0 7
[junit4:junit4]   2> 10591 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 10592 T232 C3 P21230 /update {wt=javabin&version=2} {add=[2 (1417427041547976704)]} 0 12
[junit4:junit4]   2> 10595 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 10596 T233 C3 P21230 /update {wt=javabin&version=2} {add=[10002 (1417427041551122432)]} 0 13
[junit4:junit4]   2> 10600 T218 C4 P21225 /update {wt=javabin&version=2} {add=[3 (1417427041565802496)]} 0 3
[junit4:junit4]   2> 10603 T217 C4 P21225 /update {wt=javabin&version=2} {add=[10003 (1417427041568948224)]} 0 3
[junit4:junit4]   2> 10616 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 10617 T234 C3 P21230 /update {wt=javabin&version=2} {add=[3 (1417427041574191104)]} 0 12
[junit4:junit4]   2> 10618 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10619 T232 C3 P21230 /update {wt=javabin&version=2} {add=[10003 (1417427041577336832)]} 0 11
[junit4:junit4]   2> 10622 T216 C4 P21225 /update {wt=javabin&version=2} {delete=[2 (-1417427041590968320)]} 0 1
[junit4:junit4]   2> 10624 T218 C4 P21225 /update {wt=javabin&version=2} {delete=[10002 (-1417427041594114048)]} 0 0
[junit4:junit4]   2> 10636 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 10636 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 10637 T233 C3 P21230 /update {wt=javabin&version=2} {delete=[2 (-1417427041596211200)]} 0 11
[junit4:junit4]   2> 10637 T234 C3 P21230 /update {wt=javabin&version=2} {delete=[10002 (-1417427041599356928)]} 0 8
[junit4:junit4]   2> 10645 T217 C4 P21225 /update {wt=javabin&version=2} {add=[4 (1417427041611939840)]} 0 4
[junit4:junit4]   2> 10648 T216 C4 P21225 /update {wt=javabin&version=2} {add=[10004 (1417427041612988416)]} 0 6
[junit4:junit4]   2> 10660 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10662 T232 C3 P21230 /update {wt=javabin&version=2} {add=[4 (1417427041621377024)]} 0 12
[junit4:junit4]   2> 10663 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 10664 T233 C3 P21230 /update {wt=javabin&version=2} {add=[10004 (1417427041623474176)]} 0 12
[junit4:junit4]   2> 10669 T218 C4 P21225 /update {wt=javabin&version=2} {add=[5 (1417427041638154240)]} 0 3
[junit4:junit4]   2> 10685 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 10686 T234 C3 P21230 /update {wt=javabin&version=2} {add=[5 (1417427041646542848)]} 0 12
[junit4:junit4]   2> 10698 T216 C4 P21225 /update {wt=javabin&version=2} {add=[6 (1417427041664368640)]} 0 8
[junit4:junit4]   2> 10709 T217 C4 P21225 /update {wt=javabin&version=2} {add=[10005 (1417427041640251392)]} 0 41
[junit4:junit4]   2> 10731 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=6 
[junit4:junit4]   2> 10732 T232 C3 P21230 /update {wt=javabin&version=2} {add=[6 (1417427041675902976)]} 0 30
[junit4:junit4]   2> 10739 T216 C4 P21225 /update {wt=javabin&version=2} {add=[7 (1417427041711554560)]} 0 3
[junit4:junit4]   2> 10742 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=20 
[junit4:junit4]   2> 10743 T233 C3 P21230 /update {wt=javabin&version=2} {add=[10005 (1417427041687437312)]} 0 30
[junit4:junit4]   2> 10750 T218 C4 P21225 /update {wt=javabin&version=2} {add=[10006 (1417427041723088896)]} 0 3
[junit4:junit4]   2> 10752 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10754 T232 C3 P21230 /update {wt=javabin&version=2} {add=[7 (1417427041718894592)]} 0 11
[junit4:junit4]   2> 10758 T217 C4 P21225 /update {wt=javabin&version=2} {delete=[3 (-1417427041734623232)]} 0 0
[junit4:junit4]   2> 10765 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 10766 T234 C3 P21230 /update {wt=javabin&version=2} {add=[10006 (1417427041731477504)]} 0 11
[junit4:junit4]   2> 10766 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 10767 T233 C3 P21230 /update {wt=javabin&version=2} {delete=[3 (-1417427041738817536)]} 0 5
[junit4:junit4]   2> 10773 T216 C4 P21225 /update {wt=javabin&version=2} {add=[10007 (1417427041747206144)]} 0 3
[junit4:junit4]   2> 10775 T218 C4 P21225 /update {wt=javabin&version=2} {add=[8 (1417427041749303296)]} 0 3
[junit4:junit4]   2> 10788 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 10789 T232 C3 P21230 /update {wt=javabin&version=2} {add=[10007 (1417427041754546176)]} 0 12
[junit4:junit4]   2> 10790 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 10791 T234 C3 P21230 /update {wt=javabin&version=2} {add=[8 (1417427041756643328)]} 0 12
[junit4:junit4]   2> 10793 T217 C4 P21225 /update {wt=javabin&version=2} {delete=[10003 (-1417427041771323392)]} 0 0
[junit4:junit4]   2> 10798 T216 C4 P21225 /update {wt=javabin&version=2} {add=[9 (1417427041773420544)]} 0 3
[junit4:junit4]   2> 10802 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 10803 T233 C3 P21230 /update {wt=javabin&version=2} {delete=[10003 (-1417427041775517696)]} 0 6
[junit4:junit4]   2> 10810 T218 C4 P21225 /update {wt=javabin&version=2} {add=[10008 (1417427041786003456)]} 0 3
[junit4:junit4]   2> 10812 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10814 T232 C3 P21230 /update {wt=javabin&version=2} {add=[9 (1417427041780760576)]} 0 12
[junit4:junit4]   2> 10818 T217 C4 P21225 /update {wt=javabin&version=2} {delete=[4 (-1417427041797537792)]} 0 0
[junit4:junit4]   2> 10823 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10824 T234 C3 P21230 /update {wt=javabin&version=2} {add=[10008 (1417427041793343488)]} 0 10
[junit4:junit4]   2> 10825 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 10827 T233 C3 P21230 /update {wt=javabin&version=2} {delete=[4 (-1417427041802780672)]} 0 4
[junit4:junit4]   2> 10832 T216 C4 P21225 /update {wt=javabin&version=2} {add=[10009 (1417427041809072128)]} 0 3
[junit4:junit4]   2> 10834 T218 C4 P21225 /update {wt=javabin&version=2} {add=[10 (1417427041811169280)]} 0 3
[junit4:junit4]   2> 10846 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10847 T232 C3 P21230 /update {wt=javabin&version=2} {add=[10009 (1417427041817460736)]} 0 10
[junit4:junit4]   2> 10849 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10850 T234 C3 P21230 /update {wt=javabin&version=2} {add=[10 (1417427041819557888)]} 0 11
[junit4:junit4]   2> 10851 T217 C4 P21225 /update {wt=javabin&version=2} {delete=[10004 (-1417427041832140800)]} 0 0
[junit4:junit4]   2> 10859 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 10860 T233 C3 P21230 /update {wt=javabin&version=2} {delete=[10004 (-1417427041837383680)]} 0 4
[junit4:junit4]   2> 10870 T218 C4 P21225 /update {wt=javabin&version=2} {add=[10010 (1417427041845772288)]} 0 6
[junit4:junit4]   2> 10873 T216 C4 P21225 /update {wt=javabin&version=2} {add=[11 (1417427041835286528)]} 0 19
[junit4:junit4]   2> 10902 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=6 
[junit4:junit4]   2> 10903 T232 C3 P21230 /update {wt=javabin&version=2} {add=[10010 (1417427041856258048)]} 0 29
[junit4:junit4]   2> 10905 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=20 
[junit4:junit4]   2> 10907 T234 C3 P21230 /update {wt=javabin&version=2} {add=[11 (1417427041859403776)]} 0 30
[junit4:junit4]   2> 10910 T217 C4 P21225 /update {wt=javabin&version=2} {add=[10011 (1417427041890861056)]} 0 3
[junit4:junit4]   2> 10911 T218 C4 P21225 /update {wt=javabin&version=2} {delete=[6 (-1417427041895055360)]} 0 0
[junit4:junit4]   2> 10919 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 10920 T232 C3 P21230 /update {wt=javabin&version=2} {delete=[6 (-1417427041900298240)]} 0 5
[junit4:junit4]   2> 10924 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10926 T233 C3 P21230 /update {wt=javabin&version=2} {add=[10011 (1417427041899249664)]} 0 11
[junit4:junit4]   2> 10927 T216 C4 P21225 /update {wt=javabin&version=2} {add=[12 (1417427041908686848)]} 0 3
[junit4:junit4]   2> 10930 T217 C4 P21225 /update {wt=javabin&version=2} {delete=[10006 (-1417427041914978304)]} 0 0
[junit4:junit4]   2> 10940 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 10940 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10941 T232 C3 P21230 /update {wt=javabin&version=2} {delete=[10006 (-1417427041920221184)]} 0 6
[junit4:junit4]   2> 10941 T234 C3 P21230 /update {wt=javabin&version=2} {add=[12 (1417427041916026880)]} 0 10
[junit4:junit4]   2> 10948 T218 C4 P21225 /update {wt=javabin&version=2} {add=[13 (1417427041930706944)]} 0 3
[junit4:junit4]   2> 10950 T216 C4 P21225 /update {wt=javabin&version=2} {add=[10012 (1417427041930706945)]} 0 5
[junit4:junit4]   2> 10962 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10964 T233 C3 P21230 /update {wt=javabin&version=2} {add=[13 (1417427041938046976)]} 0 12
[junit4:junit4]   2> 10965 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 10966 T232 C3 P21230 /update {wt=javabin&version=2} {add=[10012 (1417427041939095552)]} 0 14
[junit4:junit4]   2> 10971 T217 C4 P21225 /update {wt=javabin&version=2} {add=[14 (1417427041954824192)]} 0 3
[junit4:junit4]   2> 10973 T218 C4 P21225 /update {wt=javabin&version=2} {add=[10013 (1417427041956921344)]} 0 3
[junit4:junit4]   2> 10984 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10985 T233 C3 P21230 /update {wt=javabin&version=2} {add=[14 (1417427041962164224)]} 0 10
[junit4:junit4]   2> 10986 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 10987 T232 C3 P21230 /update {wt=javabin&version=2} {add=[10013 (1417427041964261376)]} 0 10
[junit4:junit4]   2> 10994 T217 C4 P21225 /update {wt=javabin&version=2} {add=[15 (1417427041978941440)]} 0 4
[junit4:junit4]   2> 10996 T216 C4 P21225 /update {wt=javabin&version=2} {add=[10014 (1417427041978941441)]} 0 5
[junit4:junit4]   2> 11007 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 11008 T234 C3 P21230 /update {wt=javabin&version=2} {add=[15 (1417427041986281472)]} 0 10
[junit4:junit4]   2> 11009 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 11010 T233 C3 P21230 /update {wt=javabin&version=2} {add=[10014 (1417427041988378624)]} 0 10
[junit4:junit4]   2> 11020 T217 C4 P21225 /update {wt=javabin&version=2} {add=[10015 (1417427042003058688)]} 0 7
[junit4:junit4]   2> 11031 T218 C4 P21225 /update {wt=javabin&version=2} {add=[16 (1417427042002010112)]} 0 18
[junit4:junit4]   2> 11053 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=7 
[junit4:junit4]   2> 11054 T232 C3 P21230 /update {wt=javabin&version=2} {add=[10015 (1417427042013544448)]} 0 30
[junit4:junit4]   2> 11061 T216 C4 P21225 /update {wt=javabin&version=2} {add=[10016 (1417427042049196032)]} 0 3
[junit4:junit4]   2> 11063 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=18 
[junit4:junit4]   2> 11064 T234 C3 P21230 /update {wt=javabin&version=2} {add=[16 (1417427042026127360)]} 0 28
[junit4:junit4]   2> 11068 T217 C4 P21225 /update {wt=javabin&version=2} {delete=[7 (-1417427042059681792)]} 0 2
[junit4:junit4]   2> 11075 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 11077 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 11077 T233 C3 P21230 /update {wt=javabin&version=2} {add=[10016 (1417427042056536064)]} 0 12
[junit4:junit4]   2> 11078 T232 C3 P21230 /update {wt=javabin&version=2} {delete=[7 (-1417427042063876096)]} 0 6
[junit4:junit4]   2> 11081 T218 C4 P21225 /update {wt=javabin&version=2} {delete=[10007 (-1417427042073313280)]} 0 0
[junit4:junit4]   2> 11085 T216 C4 P21225 /update {wt=javabin&version=2} {add=[17 (1417427042074361856)]} 0 3
[junit4:junit4]   2> 11090 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 11092 T234 C3 P21230 /update {wt=javabin&version=2} {delete=[10007 (-1417427042078556160)]} 0 6
[junit4:junit4]   2> 11098 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 11099 T217 C4 P21225 /update {wt=javabin&version=2} {add=[10017 (1417427042089041920)]} 0 3
[junit4:junit4]   2> 11100 T233 C3 P21230 /update {wt=javabin&version=2} {add=[17 (1417427042081701888)]} 0 11
[junit4:junit4]   2> 11104 T218 C4 P21225 /update {wt=javabin&version=2} {delete=[9 (-1417427042097430528)]} 0 0
[junit4:junit4]   2> 11111 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 11113 T234 C3 P21230 /update {wt=javabin&version=2} {delete=[9 (-1417427042101624832)]} 0 5
[junit4:junit4]   2> 11113 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 11114 T232 C3 P21230 /update {wt=javabin&version=2} {add=[10017 (1417427042097430528)]} 0 10
[junit4:junit4]   2> 11120 T217 C4 P21225 /update {wt=javabin&version=2} {delete=[10009 (-1417427042113159168)]} 0 1
[junit4:junit4]   2> 11121 T216 C4 P21225 /update {wt=javabin&version=2} {add=[18 (1417427042111062016)]} 0 4
[junit4:junit4]   2> 11127 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 11129 T233 C3 P21230 /update {wt=javabin&version=2} {delete=[10009 (-1417427042118402048)]} 0 5
[junit4:junit4]   2> 11136 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 11136 T218 C4 P21225 /update {wt=javabin&version=2} {add=[10018 (1417427042127839232)]} 0 3
[junit4:junit4]   2> 11137 T234 C3 P21230 /update {wt=javabin&version=2} {add=[18 (1417427042119450624)]} 0 12
[junit4:junit4]   2> 11145 T217 C4 P21225 /update {wt=javabin&version=2} {add=[19 (1417427042137276416)]} 0 4
[junit4:junit4]   2> 11152 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 11153 T233 C3 P21230 /update {wt=javabin&version=2} {add=[10018 (1417427042137276416)]} 0 12
[junit4:junit4]   2> 11159 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 11160 T234 C3 P21230 /update {wt=javabin&version=2} {add=[19 (1417427042145665024)]} 0 10
[junit4:junit4]   2> 11161 T216 C4 P21225 /update {wt=javabin&version=2} {add=[10019 (1417427042154053632)]} 0 2
[junit4:junit4]   2> 11167 T218 C4 P21225 /update {wt=javabin&version=2} {add=[20 (1417427042160345088)]} 0 3
[junit4:junit4]   2> 11174 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 11176 T232 C3 P21230 /update {wt=javabin&version=2} {add=[10019 (1417427042161393664)]} 0 11
[junit4:junit4]   2> 11180 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 11181 T233 C3 P21230 /update {wt=javabin&version=2} {add=[20 (1417427042167685120)]} 0 10
[junit4:junit4]   2> 11192 T216 C4 P21225 /update {wt=javabin&version=2} {add=[21 (1417427042183413760)]} 0 7
[junit4:junit4]   2> 11201 T217 C4 P21225 /update {wt=javabin&version=2} {add=[10020 (1417427042177122304)]} 0 21
[junit4:junit4]   2> 11226 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=5 
[junit4:junit4]   2> 11227 T232 C3 P21230 /update {wt=javabin&version=2} {add=[21 (1417427042194948096)]} 0 30
[junit4:junit4]   2> 11234 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=17 
[junit4:junit4]   2> 11235 T218 C4 P21225 /update {wt=javabin&version=2} {add=[22 (1417427042231648256)]} 0 3
[junit4:junit4]   2> 11235 T233 C3 P21230 /update {wt=javabin&version=2} {add=[10020 (1417427042205433856)]} 0 29
[junit4:junit4]   2> 11243 T216 C4 P21225 /update {wt=javabin&version=2} {add=[10021 (1417427042240036864)]} 0 3
[junit4:junit4]   2> 11251 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=5 
[junit4:junit4]   2> 11252 T234 C3 P21230 /update {wt=javabin&version=2} {add=[22 (1417427042238988288)]} 0 13
[junit4:junit4]   2> 11256 T217 C4 P21225 /update {wt=javabin&version=2} {delete=[15 (-1417427042256814080)]} 0 0
[junit4:junit4]   2> 11259 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 11260 T232 C3 P21230 /update {wt=javabin&version=2} {add=[10021 (1417427042247376896)]} 0 13
[junit4:junit4]   2> 11265 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 11266 T233 C3 P21230 /update {wt=javabin&version=2} {delete=[15 (-1417427042261008384)]} 0 6
[junit4:junit4]   2> 11267 T218 C4 P21225 /update {wt=javabin&version=2} {add=[10022 (1417427042265202688)]} 0 3
[junit4:junit4]   2> 11273 T216 C4 P21225 /update {wt=javabin&version=2} {add=[23 (1417427042271494144)]} 0 4
[junit4:junit4]   2> 11280 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 11281 T234 C3 P21230 /update {wt=javabin&version=2} {add=[10022 (1417427042273591296)]} 0 9
[junit4:junit4]   2> 11285 T217 C4 P21225 /update {wt=javabin&version=2} {delete=[10015 (-1417427042287222784)]} 0 0
[junit4:junit4]   2> 11286 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 11291 T232 C3 P21230 /update {wt=javabin&version=2} {add=[23 (1417427042278834176)]} 0 14
[junit4:junit4]   2> 11296 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 11297 T218 C4 P21225 /update {wt=javabin&version=2} {delete=[18 (-1417427042298757120)]} 0 1
[junit4:junit4]   2> 11297 T233 C3 P21230 /update {wt=javabin&version=2} {delete=[10015 (-1417427042294562816)]} 0 6
[junit4:junit4]   2> 11304 T216 C4 P21225 /update {wt=javabin&version=2} {add=[10023 (1417427042304000000)]} 0 3
[junit4:junit4]   2> 11304 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 11305 T234 C3 P21230 /update {wt=javabin&version=2} {delete=[18 (-1417427042304000000)]} 0 4
[junit4:junit4]   2> 11311 T217 C4 P21225 /update {wt=javabin&version=2} {add=[24 (1417427042312388608)]} 0 3
[junit4:junit4]   2> 11316 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 11317 T232 C3 P21230 /update {wt=javabin&version=2} {add=[10023 (1417427042311340032)]} 0 9
[junit4:junit4]   2> 11321 T218 C4 P21225 /update {wt=javabin&version=2} {delete=[10018 (-1417427042324971520)]} 0 0
[junit4:junit4]   2> 11324 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 11325 T233 C3 P21230 /update {wt=javabin&version=2} {add=[24 (1417427042319728640)]} 0 9
[junit4:junit4]   2> 11329 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 11329 T216 C4 P21225 /update {wt=javabin&version=2} {delete=[21 (-1417427042333360128)]} 0 0
[junit4:junit4]   2> 11330 T234 C3 P21230 /update {wt=javabin&version=2} {delete=[10018 (-1417427042329165824)]} 0 5
[junit4:junit4]   2> 11336 T217 C4 P21225 /update {wt=javabin&version=2} {add=[10024 (1417427042338603008)]} 0 3
[junit4:junit4]   2> 11337 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 11338 T232 C3 P21230 /update {wt=javabin&version=2} {delete=[21 (-1417427042337554432)]} 0 5
[junit4:junit4]   2> 11344 T218 C4 P21225 /update {wt=javabin&version=2} {add=[25 (1417427042346991616)]} 0 3
[junit4:junit4]   2> 11349 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 11350 T233 C3 P21230 /update {wt=javabin&version=2} {add=[10024 (1417427042345943040)]} 0 10
[junit4:junit4]   2> 11354 T216 C4 P21225 /update {wt=javabin&version=2} {delete=[10021 (-1417427042359574528)]} 0 1
[junit4:junit4]   2> 11356 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 11357 T234 C3 P21230 /update {wt=javabin&version=2} {add=[25 (1417427042353283072)]} 0 10
[junit4:junit4]   2> 11361 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 11362 T232 C3 P21230 /update {wt=javabin&version=2} {delete=[10021 (-1417427042363768832)]} 0 4
[junit4:junit4]   2> 11370 T218 C4 P21225 /update {wt=javabin&version=2} {add=[10025 (1417427042372157440)]} 0 4
[junit4:junit4]   2> 11378 T217 C4 P21225 /update {wt=javabin&version=2} {add=[26 (1417427042366914560)]} 0 17
[junit4:junit4]   2> 11392 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 11393 T233 C3 P21230 /update {wt=javabin&version=2} {add=[10025 (1417427042381594624)]} 0 18
[junit4:junit4]   2> 11399 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=10 
[junit4:junit4]   2> 11399 T234 C3 P21230 /update {wt=javabin&version=2} {add=[26 (1417427042388934656)]} 0 17
[junit4:junit4]   2> 11400 T218 C4 P21225 /update {wt=javabin&version=2} {add=[10026 (1417427042405711872)]} 0 2
[junit4:junit4]   2> 11403 T217 C4 P21225 /update {wt=javabin&version=2} {delete=[22 (-1417427042410954752)]} 0 0
[junit4:junit4]   2> 11411 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 11412 T232 C3 P21230 /update {wt=javabin&version=2} {delete=[22 (-1417427042416197632)]} 0 4
[junit4:junit4]   2> 11412 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 11413 T233 C3 P21230 /update {wt=javabin&version=2} {add=[10026 (1417427042413051904)]} 0 9
[junit4:junit4]   2> 11417 T216 C4 P21225 /update {wt=javabin&version=2} {delete=[10022 (-1417427042425634816)]} 0 1
[junit4:junit4]   2> 11418 T218 C4 P21225 /update {wt=javabin&version=2} {add=[27 (1417427042424586240)]} 0 2
[junit4:junit4]   2> 11425 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 11426 T234 C3 P21230 /update {wt=javabin&version=2} {delete=[10022 (-1417427042430877696)]} 0 4
[junit4:junit4]   2> 11430 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 11431 T232 C3 P21230 /update {wt=javabin&version=2} {add=[27 (1417427042431926272)]} 0 8
[junit4:junit4]   2> 11432 T217 C4 P21225 /update {wt=javabin&version=2} {add=[10027 (1417427042439266304)]} 0 2
[junit4:junit4]   2> 11435 T216 C4 P21225 /update {wt=javabin&version=2} {delete=[26 (-1417427042444509184)]} 0 0
[junit4:junit4]   2> 11442 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 11443 T234 C3 P21230 /update {wt=javabin&version=2} {delete=[26 (-1417427042447654912)]} 0 5
[junit4:junit4]   2> 11443 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 11444 T233 C3 P21230 /update {wt=javabin&version=2} {add=[10027 (1417427042446606336)]} 0 8
[junit4:junit4]   2> 11448 T217 C4 P21225 /update {wt=javabin&version=2} {delete=[10026 (-1417427042458140672)]} 0 0
[junit4:junit4]   2> 11450 T218 C4 P21225 /update {wt=javabin&version=2} {add=[28 (1417427042457092096)]} 0 3
[junit4:junit4]   2> 11455 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 11456 T232 C3 P21230 /update {wt=javabin&version=2} {delete=[10026 (-1417427042462334976)]} 0 5
[junit4:junit4]   2> 11460 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 11461 T234 C3 P21230 /update {wt=javabin&version=2} {add=[28 (1417427042464432128)]} 0 8
[junit4:junit4]   2> 11462 T216 C4 P21225 /update {wt=javabin&version=2} {add=[10028 (1417427042470723584)]} 0 2
[junit4:junit4]   2> 11465 T217 C4 P21225 /update {wt=javabin&version=2} {delete=[27 (-1417427042475966464)]} 0 0
[junit4:junit4]   2> 11472 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 11473 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 11473 T233 C3 P21230 /update {wt=javabin&version=2} {add=[10028 (1417427042477015040)]} 0 8
[junit4:junit4]   2> 11474 T232 C3 P21230 /update {wt=javabin&version=2} {delete=[27 (-1417427042481209344)]} 0 5
[junit4:junit4]   2> 11477 T218 C4 P21225 /update {wt=javabin&version=2} {delete=[10027 (-1417427042488549376)]} 0 0
[junit4:junit4]   2> 11479 T216 C4 P21225 /update {wt=javabin&version=2} {add=[29 (1417427042489597952)]} 0 2
[junit4:junit4]   2> 11484 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 11485 T234 C3 P21230 /update {wt=javabin&version=2} {delete=[10027 (-1417427042492743680)]} 0 5
[junit4:junit4]   2> 11491 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 11491 T217 C4 P21225 /update {wt=javabin&version=2} {add=[10029 (1417427042501132288)]} 0 2
[junit4:junit4]   2> 11493 T233 C3 P21230 /update {wt=javabin&version=2} {add=[29 (1417427042494840832)]} 0 11
[junit4:junit4]   2> 11496 T218 C4 P21225 /update {wt=javabin&version=2} {delete=[29 (-1417427042508472320)]} 0 0
[junit4:junit4]   2> 11503 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 11504 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 11505 T232 C3 P21230 /update {wt=javabin&version=2} {add=[10029 (1417427042508472320)]} 0 9
[junit4:junit4]   2> 11505 T234 C3 P21230 /update {wt=javabin&version=2} {delete=[29 (-1417427042513715200)]} 0 5
[junit4:junit4]   2> 11509 T218 C4 P21225 /update {wt=javabin&version=2} {delete=[10029 (-1417427042522103808)]} 0 0
[junit4:junit4]   2> 11511 T217 C4 P21225 /update {wt=javabin&version=2} {add=[30 (1417427042522103809)]} 0 2
[junit4:junit4]   2> 11516 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 11517 T233 C3 P21230 /update {wt=javabin&version=2} {delete=[10029 (-1417427042526298112)]} 0 4
[junit4:junit4]   2> 11522 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 11523 T232 C3 P21230 /update {wt=javabin&version=2} {add=[30 (1417427042529443840)]} 0 7
[junit4:junit4]   2> 11532 T218 C4 P21225 /update {wt=javabin&version=2} {add=[31 (1417427042542026752)]} 0 4
[junit4:junit4]   2> 11538 T216 C4 P21225 /update {wt=javabin&version=2} {add=[10030 (1417427042534686720)]} 0 17
[junit4:junit4]   2> 11562 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=12 
[junit4:junit4]   2> 11562 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=5 
[junit4:junit4]   2> 11563 T232 C3 P21230 /update {wt=javabin&version=2} {add=[10030 (1417427042556706816)]} 0 21
[junit4:junit4]   2> 11563 T234 C3 P21230 /update {wt=javabin&version=2} {add=[31 (1417427042551463936)]} 0 26
[junit4:junit4]   2> 11568 T218 C4 P21225 /update {wt=javabin&version=2} {delete=[31 (-1417427042583969792)]} 0 1
[junit4:junit4]   2> 11569 T217 C4 P21225 /update {wt=javabin&version=2} {add=[10031 (1417427042582921216)]} 0 2
[junit4:junit4]   2> 11576 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 11577 T233 C3 P21230 /update {wt=javabin&version=2} {delete=[31 (-1417427042588164096)]} 0 6
[junit4:junit4]   2> 11581 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 11582 T232 C3 P21230 /update {wt=javabin&version=2} {add=[10031 (1417427042590261248)]} 0 9
[junit4:junit4]   2> 11583 T216 C4 P21225 /update {wt=javabin&version=2} {add=[32 (1417427042597601280)]} 0 3
[junit4:junit4]   2> 11586 T218 C4 P21225 /update {wt=javabin&version=2} {delete=[10031 (-1417427042602844160)]} 0 1
[junit4:junit4]   2> 11593 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 11594 T233 C3 P21230 /update {wt=javabin&version=2} {delete=[10031 (-1417427042607038464)]} 0 4
[junit4:junit4]   2> 11595 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 11596 T234 C3 P21230 /update {wt=javabin&version=2} {add=[32 (1417427042603892736)]} 0 10
[junit4:junit4]   2> 11600 T217 C4 P21225 /update {wt=javabin&version=2} {add=[10032 (1417427042615427072)]} 0 2
[junit4:junit4]   2> 11602 T216 C4 P21225 /update {wt=javabin&version=2} {add=[33 (1417427042617524224)]} 0 3
[junit4:junit4]   2> 11611 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 11612 T232 C3 P21230 /update {wt=javabin&version=2} {add=[10032 (1417427042621718528)]} 0 9
[junit4:junit4]   2> 11613 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 11614 T233 C3 P21230 /update {wt=javabin&version=2} {add=[33 (1417427042623815680)]} 0 9
[junit4:junit4]   2> 11617 T218 C4 P21225 /update {wt=javabin&version=2} {add=[10033 (1417427042634301440)]} 0 2
[junit4:junit4]   2> 11620 T217 C4 P21225 /update {wt=javabin&version=2} {add=[34 (1417427042636398592)]} 0 3
[junit4:junit4]   2> 11628 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 11629 T234 C3 P21230 /update {wt=javabin&version=2} {add=[10033 (1417427042640592896)]} 0 8
[junit4:junit4]   2> 11632 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 11633 T232 C3 P21230 /update {wt=javabin&version=2} {add=[34 (1417427042642690048)]} 0 9
[junit4:junit4]   2> 11635 T216 C4 P21225 /update {wt=javabin&version=2} {add=[10034 (1417427042652127232)]} 0 2
[junit4:junit4]   2> 11637 T218 C4 P21225 /update {wt=javabin&version=2} {delete=[34 (-1417427042656321536)]} 0 1
[junit4:junit4]   2> 11644 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 11646 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 11646 T234 C3 P21230 /update {wt=javabin&version=2} {delete=[34 (-1417427042660515840)]} 0 5
[junit4:junit4]   2> 11647 T233 C3 P21230 /update {wt=javabin&version=2} {add=[10034 (1417427042658418688)]} 0 9
[junit4:junit4]   2> 11652 T216 C4 P21225 /update {wt=javabin&version=2} {delete=[10034 (-1417427042672050176)]} 0 0
[junit4:junit4]   2> 11653 T217 C4 P21225 /update {wt=javabin&version=2} {add=[35 (1417427042671001600)]} 0 3
[junit4:junit4]   2> 11660 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 11661 T232 C3 P21230 /update {wt=javabin&version=2} {delete=[10034 (-1417427042676244480)]} 0 5
[junit4:junit4]   2> 11664 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 11665 T234 C3 P21230 /update {wt=javabin&version=2} {add=[35 (1417427042677293056)]} 0 8
[junit4:junit4]   2> 11673 T216 C4 P21225 /update {wt=javabin&version=2} {add=[36 (1417427042689875968)]} 0 4
[junit4:junit4]   2> 11676 T218 C4 P21225 /update {wt=javabin&version=2} {add=[10035 (1417427042685681664)]} 0 11
[junit4:junit4]   2> 11701 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 11702 T233 C3 P21230 /update {wt=javabin&version=2} {add=[36 (1417427042699313152)]} 0 24
[junit4:junit4]   2> 11705 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=16 
[junit4:junit4]   2> 11707 T232 C3 P21230 /update {wt=javabin&version=2} {add=[10035 (1417427042703507456)]} 0 26
[junit4:junit4]   2> 11710 T217 C4 P21225 /update {wt=javabin&version=2} {add=[37 (1417427042729721856)]} 0 3
[junit4:junit4]   2> 11713 T216 C4 P21225 /update {wt=javabin&version=2} {add=[10036 (1417427042733916160)]} 0 2
[junit4:junit4]   2> 11721 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 11722 T234 C3 P21230 /update {wt=javabin&version=2} {add=[37 (1417427042737061888)]} 0 8
[junit4:junit4]   2> 11724 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 11726 T233 C3 P21230 /update {wt=javabin&version=2} {add=[10036 (1417427042740207616)]} 0 9
[junit4:junit4]   2> 11726 T218 C4 P21225 /update {wt=javabin&version=2} {delete=[36 (-1417427042749644800)]} 0 1
[junit4:junit4]   2> 11732 T217 C4 P21225 /update {wt=javabin&version=2} {add=[10037 (1417427042753839104)]} 0 2
[junit4:junit4]   2> 11734 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 11735 T232 C3 P21230 /update {wt=javabin&version=2} {delete=[36 (-1417427042753839104)]} 0 5
[junit4:junit4]   2> 11741 T216 C4 P21225 /update {wt=javabin&version=2} {add=[38 (1417427042763276288)]} 0 2
[junit4:junit4]   2> 11741 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 11743 T234 C3 P21230 /update {wt=javabin&version=2} {add=[10037 (1417427042760130560)]} 0 8
[junit4:junit4]   2> 11746 T218 C4 P21225 /update {wt=javabin&version=2} {delete=[10036 (-1417427042770616320)]} 0 1
[junit4:junit4]   2> 11753 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 11754 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 11754 T232 C3 P21230 /update {wt=javabin&version=2} {delete=[10036 (-1417427042774810624)]} 0 4
[junit4:junit4]   2> 11755 T233 C3 P21230 /update {wt=javabin&version=2} {add=[38 (1417427042770616320)]} 0 9
[junit4:junit4]   2> 11761 T216 C4 P21225 /update {wt=javabin&version=2} {add=[39 (1417427042784247808)]} 0 2
[junit4:junit4]   2> 11762 T217 C4 P21225 /update {wt=javabin&version=2} {add=[10038 (1417427042784247809)]} 0 3
[junit4:junit4]   2> 11771 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 11772 T234 C3 P21230 /update {wt=javabin&version=2} {add=[39 (1417427042790539264)]} 0 7
[junit4:junit4]   2> 11773 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 11774 T232 C3 P21230 /update {wt=javabin&version=2} {add=[10038 (1417427042791587840)]} 0 9
[junit4:junit4]   2> 11776 T218 C4 P21225 /update {wt=javabin&version=2} {delete=[39 (-1417427042802073600)]} 0 1
[junit4:junit4]   2> 11779 T216 C4 P21225 /update {wt=javabin&version=2} {add=[10039 (1417427042804170752)]} 0 2
[junit4:junit4]   2> 11783 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 11783 T233 C3 P21230 /update {wt=javabin&version=2} {delete=[39 (-1417427042805219328)]} 0 4
[junit4:junit4]   2> 11788 T217 C4 P21225 /update {wt=javabin&version=2} {add=[40 (1417427042813607936)]} 0 2
[junit4:junit4]   2> 11788 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 11789 T234 C3 P21230 /update {wt=javabin&version=2} {add=[10039 (1417427042809413632)]} 0 7
[junit4:junit4]   2> 11793 T218 C4 P21225 /update {wt=javabin&version=2} {delete=[10039 (-1417427042818850816)]} 0 1
[junit4:junit4]   2> 11797 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 11798 T233 C3 P21230 /update {wt=javabin&version=2} {add=[40 (1417427042818850816)]} 0 7
[junit4:junit4]   2> 11800 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 11803 T232 C3 P21230 /update {wt=javabin&version=2} {delete=[10039 (-1417427042824093696)]} 0 7
[junit4:junit4]   2> 11811 T218 C4 P21225 /update {wt=javabin&version=2} {add=[10040 (1417427042835628032)]} 0 4
[junit4:junit4]   2> 11959 T216 C4 P21225 /update {wt=javabin&version=2} {add=[41 (1417427042832482304)]} 0 155
[junit4:junit4]   2> 11973 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 11974 T232 C3 P21230 /update {wt=javabin&version=2} {add=[41 (1417427042998157312)]} 0 12
[junit4:junit4]   2> 11982 T217 C4 P21225 /update {wt=javabin&version=2} {add=[42 (1417427043014934528)]} 0 3
[junit4:junit4]   2> 11992 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 11993 T233 C3 P21230 /update {wt=javabin&version=2} {add=[42 (1417427043022274560)]} 0 8
[junit4:junit4]   2> 11997 T218 C4 P21225 /update {wt=javabin&version=2} {delete=[42 (-1417427043033808896)]} 0 0
[junit4:junit4]   2> 12004 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 12005 T232 C3 P21230 /update {wt=javabin&version=2} {delete=[42 (-1417427043038003200)]} 0 5
[junit4:junit4]   2> 12010 T216 C4 P21225 /update {wt=javabin&version=2} {add=[43 (1417427043046391808)]} 0 2
[junit4:junit4]   2> 12019 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 12020 T233 C3 P21230 /update {wt=javabin&version=2} {add=[43 (1417427043051634688)]} 0 7
[junit4:junit4]   2> 12026 T217 C4 P21225 /update {wt=javabin&version=2} {add=[44 (1417427043062120448)]} 0 3
[junit4:junit4]   2> 12035 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 12036 T232 C3 P21230 /update {wt=javabin&version=2} {add=[44 (1417427043068411904)]} 0 7
[junit4:junit4]   2> 12041 T218 C4 P21225 /update {wt=javabin&version=2} {add=[45 (1417427043078897664)]} 0 2
[junit4:junit4]   2> 12050 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 12051 T233 C3 P21230 /update {wt=javabin&version=2} {add=[45 (1417427043084140544)]} 0 7
[junit4:junit4]   2> 12056 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=95 
[junit4:junit4]   2> 12056 T216 C4 P21225 /update {wt=javabin&version=2} {add=[46 (1417427043094626304)]} 0 2
[junit4:junit4]   2> 12057 T234 C3 P21230 /update {wt=javabin&version=2} {add=[10040 (1417427042842968064)]} 0 243
[junit4:junit4]   2> 12062 T217 C4 P21225 /update {wt=javabin&version=2} {add=[10041 (1417427043100917760)]} 0 2
[junit4:junit4]   2> 12065 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 12066 T232 C3 P21230 /update {wt=javabin&version=2} {add=[46 (1417427043099869184)]} 0 7
[junit4:junit4]   2> 12070 T218 C4 P21225 /update {wt=javabin&version=2} {delete=[45 (-1417427043110354944)]} 0 1
[junit4:junit4]   2> 12072 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 12073 T233 C3 P21230 /update {wt=javabin&version=2} {add=[10041 (1417427043106160640)]} 0 8
[junit4:junit4]   2> 12077 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 12078 T234 C3 P21230 /update {wt=javabin&version=2} {delete=[45 (-1417427043114549248)]} 0 5
[junit4:junit4]   2> 12078 T216 C4 P21225 /update {wt=javabin&version=2} {add=[10042 (1417427043117694976)]} 0 2
[junit4:junit4]   2> 12083 T217 C4 P21225 /update {wt=javabin&version=2} {add=[47 (1417427043122937856)]} 0 2
[junit4:junit4]   2> 12089 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 12090 T232 C3 P21230 /update {wt=javabin&version=2} {add=[10042 (1417427043122937856)]} 0 8
[junit4:junit4]   2> 12093 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 12094 T233 C3 P21230 /update {wt=javabin&version=2} {add=[47 (1417427043128180736)]} 0 8
[junit4:junit4]   2> 12094 T218 C4 P21225 /update {wt=javabin&version=2} {delete=[10042 (-1417427043135520768)]} 0 1
[junit4:junit4]   2> 12101 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 12102 T234 C3 P21230 /update {wt=javabin&version=2} {delete=[10042 (-1417427043139715072)]} 0 5
[junit4:junit4]   2> 12109 T216 C4 P21225 /update {wt=javabin&version=2} {add=[48 (1417427043139715072)]} 0 12
[junit4:junit4]   2> 12111 T217 C4 P21225 /update {wt=javabin&version=2} {add=[10043 (1417427043148103680)]} 0 5
[junit4:junit4]   2> 12132 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=4 
[junit4:junit4]   2> 12134 T232 C3 P21230 /update {wt=javabin&version=2} {add=[48 (1417427043156492288)]} 0 21
[junit4:junit4]   2> 12134 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=13 
[junit4:junit4]   2> 12135 T233 C3 P21230 /update {wt=javabin&version=2} {add=[10043 (1417427043157540864)]} 0 20
[junit4:junit4]   2> 12139 T218 C4 P21225 /update {wt=javabin&version=2} {add=[49 (1417427043181658112)]} 0 2
[junit4:junit4]   2> 12141 T216 C4 P21225 /update {wt=javabin&version=2} {add=[10044 (1417427043182706688)]} 0 2
[junit4:junit4]   2> 12151 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 12152 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 12152 T232 C3 P21230 /update {wt=javabin&version=2} {add=[10044 (1417427043187949568)]} 0 9
[junit4:junit4]   2> 12153 T234 C3 P21230 /update {wt=javabin&version=2} {add=[49 (1417427043187949569)]} 0 10
[junit4:junit4]   2> 12159 T218 C4 P21225 /update {wt=javabin&version=2} {delete=[47 (-1417427043203678209)]} 0 0
[junit4:junit4]   2> 12160 T217 C4 P21225 /update {wt=javabin&version=2} {add=[10045 (1417427043203678208)]} 0 2
[junit4:junit4]   2> 12207 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 12208 T233 C3 P21230 /update {wt=javabin&version=2} {delete=[47 (-1417427043249815552)]} 0 5
[junit4:junit4]   2> 12210 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 12211 T234 C3 P21230 /update {wt=javabin&version=2} {add=[10045 (1417427043250864128)]} 0 7
[junit4:junit4]   2> 12213 T216 C4 P21225 /update {wt=javabin&version=2} {add=[50 (1417427043259252736)]} 0 2
[junit4:junit4]   2> 12216 T217 C4 P21225 /update {wt=javabin&version=2} {add=[10046 (1417427043262398464)]} 0 2
[junit4:junit4]   2> 12222 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 12223 T232 C3 P21230 /update {wt=javabin&version=2} {add=[50 (1417427043263447040)]} 0 7
[junit4:junit4]   2> 12225 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 12226 T233 C3 P21230 /update {wt=javabin&version=2} {add=[10046 (1417427043267641344)]} 0 7
[junit4:junit4]   2> 12227 T216 C4 P21225 /update {wt=javabin&version=2} {delete=[50 (-1417427043273932800)]} 0 1
[junit4:junit4]   2> 12230 T218 C4 P21225 /update {wt=javabin&version=2} {delete=[10045 (-1417427043277078528)]} 0 1
[junit4:junit4]   2> 12233 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 12233 T234 C3 P21230 /update {wt=javabin&version=2} {delete=[50 (-1417427043277078528)]} 0 4
[junit4:junit4]   2> 12236 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 12237 T232 C3 P21230 /update {wt=javabin&version=2} {delete=[10045 (-1417427043280224256)]} 0 4
[junit4:junit4]   2> 12238 T217 C4 P21225 /update {wt=javabin&version=2} {add=[51 (1417427043284418560)]} 0 2
[junit4:junit4]   2> 12241 T216 C4 P21225 /update {wt=javabin&version=2} {add=[10047 (1417427043288612864)]} 0 2
[junit4:junit4]   2> 12247 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 12248 T233 C3 P21230 /update {wt=javabin&version=2} {add=[51 (1417427043289661440)]} 0 7
[junit4:junit4]   2> 12250 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 12250 T234 C3 P21230 /update {wt=javabin&version=2} {add=[10047 (1417427043292807168)]} 0 6
[junit4:junit4]   2> 12252 T218 C4 P21225 /update {wt=javabin&version=2} {delete=[51 (-1417427043300147200)]} 0 1
[junit4:junit4]   2> 12255 T217 C4 P21225 /update {wt=javabin&version=2} {add=[10048 (1417427043302244352)]} 0 2
[junit4:junit4]   2> 12259 T248 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 12260 T232 C3 P21230 /update {wt=javabin&version=2} {delete=[51 (-1417427043304341504)]} 0 5
[junit4:junit4]   2> 12265 T249 C2 P21235 REQ /update {distrib.from=http://127.0.0.1:21230/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 12266 T233 C3 P21230 /update {wt=javabin&version=2} {add=[10048 (1417427043308535808)]} 0 8
[junit4:junit4]   2> 12272 T218 C4 P21225 /update {wt=javabin&version=2} {add=[10049 (1417427043320070144)]} 0 3
[junit4:ju

[...truncated too long message...]

.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    7) Thread[id=339, name=qtp551926596-339 Acceptor3 SelectChannelConnector@0.0.0.0:57705, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    8) Thread[id=404, name=qtp680254087-404 Acceptor0 SelectChannelConnector@0.0.0.0:22157, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([10754BD6AFFFB7CB]:0)
[junit4:junit4]    > Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4]    >    1) Thread[id=407, name=qtp680254087-407 Acceptor3 SelectChannelConnector@0.0.0.0:22157, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=406, name=qtp680254087-406 Acceptor2 SelectChannelConnector@0.0.0.0:22157, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=405, name=qtp680254087-405 Acceptor1 SelectChannelConnector@0.0.0.0:22157, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=338, name=qtp551926596-338 Acceptor2 SelectChannelConnector@0.0.0.0:57705, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    5) Thread[id=336, name=qtp551926596-336 Acceptor0 SelectChannelConnector@0.0.0.0:57705, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    6) Thread[id=337, name=qtp551926596-337 Acceptor1 SelectChannelConnector@0.0.0.0:57705, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    7) Thread[id=339, name=qtp551926596-339 Acceptor3 SelectChannelConnector@0.0.0.0:57705, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    8) Thread[id=404, name=qtp680254087-404 Acceptor0 SelectChannelConnector@0.0.0.0:22157, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([10754BD6AFFFB7CB]:0)
[junit4:junit4] Completed on J0 in 710.58s, 1 test, 2 errors <<< FAILURES!

[...truncated 89 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:335: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/module-build.xml:61: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:1163: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:827: There were test failures: 233 suites, 965 tests, 4 suite-level errors, 757 ignored

Total time: 32 minutes 26 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure