You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2012/11/02 19:10:08 UTC

[JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 608 - Still Failing

Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/608/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.request.TestRemoteStreaming

Error Message:
4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming:     1) Thread[id=297, name=qtp1321261771-297 Acceptor2 SelectChannelConnector@0.0.0.0:58147, state=RUNNABLE, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=298, name=qtp1321261771-298 Acceptor3 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=296, name=qtp1321261771-296 Acceptor1 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=295, name=qtp1321261771-295 Acceptor0 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming: 
   1) Thread[id=297, name=qtp1321261771-297 Acceptor2 SelectChannelConnector@0.0.0.0:58147, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=298, name=qtp1321261771-298 Acceptor3 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=296, name=qtp1321261771-296 Acceptor1 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=295, name=qtp1321261771-295 Acceptor0 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([8E53F6B1A0D0F2CF]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.request.TestRemoteStreaming

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=297, name=qtp1321261771-297 Acceptor2 SelectChannelConnector@0.0.0.0:58147, state=RUNNABLE, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=298, name=qtp1321261771-298 Acceptor3 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=296, name=qtp1321261771-296 Acceptor1 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=295, name=qtp1321261771-295 Acceptor0 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=297, name=qtp1321261771-297 Acceptor2 SelectChannelConnector@0.0.0.0:58147, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=298, name=qtp1321261771-298 Acceptor3 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=296, name=qtp1321261771-296 Acceptor1 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=295, name=qtp1321261771-295 Acceptor0 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([8E53F6B1A0D0F2CF]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.update.SolrCmdDistributorTest

Error Message:
20 threads leaked from SUITE scope at org.apache.solr.update.SolrCmdDistributorTest:     1) Thread[id=32, name=qtp1160133474-32 Acceptor0 SelectChannelConnector@0.0.0.0:11368, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=62, name=qtp888776338-62 Acceptor2 SelectChannelConnector@0.0.0.0:13188, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=19, name=qtp2137918572-19 Acceptor1 SelectChannelConnector@0.0.0.0:61902, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=74, name=qtp1946214468-74 Acceptor0 SelectChannelConnector@0.0.0.0:65108, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=60, name=qtp888776338-60 Acceptor0 SelectChannelConnector@0.0.0.0:13188, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=48, name=qtp544144105-48 Acceptor2 SelectChannelConnector@0.0.0.0:53691, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=76, name=qtp1946214468-76 Acceptor2 SelectChannelConnector@0.0.0.0:65108, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=61, name=qtp888776338-61 Acceptor1 SelectChannelConnector@0.0.0.0:13188, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=77, name=qtp1946214468-77 Acceptor3 SelectChannelConnector@0.0.0.0:65108, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=63, name=qtp888776338-63 Acceptor3 SelectChannelConnector@0.0.0.0:13188, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=20, name=qtp2137918572-20 Acceptor2 SelectChannelConnector@0.0.0.0:61902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=46, name=qtp544144105-46 Acceptor0 SelectChannelConnector@0.0.0.0:53691, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=49, name=qtp544144105-49 Acceptor3 SelectChannelConnector@0.0.0.0:53691, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=33, name=qtp1160133474-33 Acceptor1 SelectChannelConnector@0.0.0.0:11368, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=47, name=qtp544144105-47 Acceptor1 SelectChannelConnector@0.0.0.0:53691, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=35, name=qtp1160133474-35 Acceptor3 SelectChannelConnector@0.0.0.0:11368, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=75, name=qtp1946214468-75 Acceptor1 SelectChannelConnector@0.0.0.0:65108, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=21, name=qtp2137918572-21 Acceptor3 SelectChannelConnector@0.0.0.0:61902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=18, name=qtp2137918572-18 Acceptor0 SelectChannelConnector@0.0.0.0:61902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=34, name=qtp1160133474-34 Acceptor2 SelectChannelConnector@0.0.0.0:11368, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 20 threads leaked from SUITE scope at org.apache.solr.update.SolrCmdDistributorTest: 
   1) Thread[id=32, name=qtp1160133474-32 Acceptor0 SelectChannelConnector@0.0.0.0:11368, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=62, name=qtp888776338-62 Acceptor2 SelectChannelConnector@0.0.0.0:13188, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=19, name=qtp2137918572-19 Acceptor1 SelectChannelConnector@0.0.0.0:61902, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=74, name=qtp1946214468-74 Acceptor0 SelectChannelConnector@0.0.0.0:65108, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=60, name=qtp888776338-60 Acceptor0 SelectChannelConnector@0.0.0.0:13188, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=48, name=qtp544144105-48 Acceptor2 SelectChannelConnector@0.0.0.0:53691, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=76, name=qtp1946214468-76 Acceptor2 SelectChannelConnector@0.0.0.0:65108, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=61, name=qtp888776338-61 Acceptor1 SelectChannelConnector@0.0.0.0:13188, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=77, name=qtp1946214468-77 Acceptor3 SelectChannelConnector@0.0.0.0:65108, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=63, name=qtp888776338-63 Acceptor3 SelectChannelConnector@0.0.0.0:13188, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=20, name=qtp2137918572-20 Acceptor2 SelectChannelConnector@0.0.0.0:61902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=46, name=qtp544144105-46 Acceptor0 SelectChannelConnector@0.0.0.0:53691, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=49, name=qtp544144105-49 Acceptor3 SelectChannelConnector@0.0.0.0:53691, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=33, name=qtp1160133474-33 Acceptor1 SelectChannelConnector@0.0.0.0:11368, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=47, name=qtp544144105-47 Acceptor1 SelectChannelConnector@0.0.0.0:53691, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=35, name=qtp1160133474-35 Acceptor3 SelectChannelConnector@0.0.0.0:11368, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=75, name=qtp1946214468-75 Acceptor1 SelectChannelConnector@0.0.0.0:65108, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=21, name=qtp2137918572-21 Acceptor3 SelectChannelConnector@0.0.0.0:61902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=18, name=qtp2137918572-18 Acceptor0 SelectChannelConnector@0.0.0.0:61902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=34, name=qtp1160133474-34 Acceptor2 SelectChannelConnector@0.0.0.0:11368, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([8E53F6B1A0D0F2CF]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.update.SolrCmdDistributorTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=32, name=qtp1160133474-32 Acceptor0 SelectChannelConnector@0.0.0.0:11368, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=19, name=qtp2137918572-19 Acceptor1 SelectChannelConnector@0.0.0.0:61902, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=62, name=qtp888776338-62 Acceptor2 SelectChannelConnector@0.0.0.0:13188, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=60, name=qtp888776338-60 Acceptor0 SelectChannelConnector@0.0.0.0:13188, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=74, name=qtp1946214468-74 Acceptor0 SelectChannelConnector@0.0.0.0:65108, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=48, name=qtp544144105-48 Acceptor2 SelectChannelConnector@0.0.0.0:53691, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=76, name=qtp1946214468-76 Acceptor2 SelectChannelConnector@0.0.0.0:65108, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=61, name=qtp888776338-61 Acceptor1 SelectChannelConnector@0.0.0.0:13188, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=20, name=qtp2137918572-20 Acceptor2 SelectChannelConnector@0.0.0.0:61902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=63, name=qtp888776338-63 Acceptor3 SelectChannelConnector@0.0.0.0:13188, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=77, name=qtp1946214468-77 Acceptor3 SelectChannelConnector@0.0.0.0:65108, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=46, name=qtp544144105-46 Acceptor0 SelectChannelConnector@0.0.0.0:53691, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=49, name=qtp544144105-49 Acceptor3 SelectChannelConnector@0.0.0.0:53691, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=33, name=qtp1160133474-33 Acceptor1 SelectChannelConnector@0.0.0.0:11368, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=47, name=qtp544144105-47 Acceptor1 SelectChannelConnector@0.0.0.0:53691, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=35, name=qtp1160133474-35 Acceptor3 SelectChannelConnector@0.0.0.0:11368, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=75, name=qtp1946214468-75 Acceptor1 SelectChannelConnector@0.0.0.0:65108, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=21, name=qtp2137918572-21 Acceptor3 SelectChannelConnector@0.0.0.0:61902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=18, name=qtp2137918572-18 Acceptor0 SelectChannelConnector@0.0.0.0:61902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=34, name=qtp1160133474-34 Acceptor2 SelectChannelConnector@0.0.0.0:11368, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=32, name=qtp1160133474-32 Acceptor0 SelectChannelConnector@0.0.0.0:11368, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=19, name=qtp2137918572-19 Acceptor1 SelectChannelConnector@0.0.0.0:61902, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=62, name=qtp888776338-62 Acceptor2 SelectChannelConnector@0.0.0.0:13188, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=60, name=qtp888776338-60 Acceptor0 SelectChannelConnector@0.0.0.0:13188, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=74, name=qtp1946214468-74 Acceptor0 SelectChannelConnector@0.0.0.0:65108, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=48, name=qtp544144105-48 Acceptor2 SelectChannelConnector@0.0.0.0:53691, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=76, name=qtp1946214468-76 Acceptor2 SelectChannelConnector@0.0.0.0:65108, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=61, name=qtp888776338-61 Acceptor1 SelectChannelConnector@0.0.0.0:13188, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=20, name=qtp2137918572-20 Acceptor2 SelectChannelConnector@0.0.0.0:61902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=63, name=qtp888776338-63 Acceptor3 SelectChannelConnector@0.0.0.0:13188, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=77, name=qtp1946214468-77 Acceptor3 SelectChannelConnector@0.0.0.0:65108, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=46, name=qtp544144105-46 Acceptor0 SelectChannelConnector@0.0.0.0:53691, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=49, name=qtp544144105-49 Acceptor3 SelectChannelConnector@0.0.0.0:53691, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=33, name=qtp1160133474-33 Acceptor1 SelectChannelConnector@0.0.0.0:11368, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=47, name=qtp544144105-47 Acceptor1 SelectChannelConnector@0.0.0.0:53691, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=35, name=qtp1160133474-35 Acceptor3 SelectChannelConnector@0.0.0.0:11368, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=75, name=qtp1946214468-75 Acceptor1 SelectChannelConnector@0.0.0.0:65108, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=21, name=qtp2137918572-21 Acceptor3 SelectChannelConnector@0.0.0.0:61902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=18, name=qtp2137918572-18 Acceptor0 SelectChannelConnector@0.0.0.0:61902, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=34, name=qtp1160133474-34 Acceptor2 SelectChannelConnector@0.0.0.0:11368, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([8E53F6B1A0D0F2CF]:0)




Build Log:
[...truncated 8368 lines...]
[junit4:junit4] Suite: org.apache.solr.update.SolrCmdDistributorTest
[junit4:junit4]   2> 213 T11 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> 674 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 982 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:61902
[junit4:junit4]   2> 1127 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1128 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 1129 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 1766 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1770 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1771 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 1803 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 1808 T11 oasc.CoreContainer.<init> New CoreContainer 665935931
[junit4:junit4]   2> 1809 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 1810 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 1897 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 1980 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 1981 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 1983 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 1984 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 2067 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 2234 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 2264 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 2338 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 4426 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 4476 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 4482 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 4920 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4942 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4948 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4970 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5078 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/control/data/
[junit4:junit4]   2> 5078 T11 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 5142 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/control/data/index/
[junit4:junit4]   2> 5143 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1351879716743/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5232 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/control/data/index forceNew:false
[junit4:junit4]   2> 5655 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@555a7085 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5dcd1c83; maxCacheMB=0.2607421875 maxMergeSizeMB=0.7861328125)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5656 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 6654 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 6678 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 6715 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 6721 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 6897 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 6997 T11 oass.SolrIndexSearcher.<init> Opening Searcher@53a46b20 main
[junit4:junit4]   2> 6998 T11 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 7117 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 7117 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 7215 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 7216 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 7217 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 7217 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 7218 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 7219 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 7219 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 7220 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 7220 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 7270 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 7791 T26 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@53a46b20 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 7793 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 7794 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 7794 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 7845 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 7869 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 7872 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:11368
[junit4:junit4]   2> 7873 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 7874 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 7875 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 7908 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 7909 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 7910 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 7910 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 7911 T11 oasc.CoreContainer.<init> New CoreContainer 97269771
[junit4:junit4]   2> 7912 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 7913 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 7958 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 7991 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 7993 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 7995 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 7996 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 8043 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 8114 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 8115 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 8169 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 8801 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 8818 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 8824 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 8848 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8854 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8859 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8864 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8868 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard0/data/
[junit4:junit4]   2> 8869 T11 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 8871 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard0/data/index/
[junit4:junit4]   2> 8873 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard0/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 8897 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard0/data/index forceNew:false
[junit4:junit4]   2> 8906 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@614c0482 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c7e3f6b),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 8906 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 8908 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 8909 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 8909 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 8910 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 8919 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 8924 T11 oass.SolrIndexSearcher.<init> Opening Searcher@41e9d96a main
[junit4:junit4]   2> 8925 T11 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 8927 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 8927 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 8928 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 8929 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 8929 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 8930 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 8930 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 8931 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 8931 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 8932 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 8932 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 8933 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 8946 T40 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@41e9d96a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 8946 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 8947 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 8948 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 8949 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 8960 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 8963 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:53691
[junit4:junit4]   2> 8964 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 8965 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 8965 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 8999 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 9000 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 9000 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 9001 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 9002 T11 oasc.CoreContainer.<init> New CoreContainer 1533846421
[junit4:junit4]   2> 9003 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 9003 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 9039 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 9059 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 9060 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 9062 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 9063 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 9110 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 9170 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 9171 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 9225 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 9831 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 9845 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 9848 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 9871 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9876 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9881 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9884 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9886 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard1/data/
[junit4:junit4]   2> 9887 T11 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 9888 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard1/data/index/
[junit4:junit4]   2> 9889 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard1/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 10008 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard1/data/index forceNew:false
[junit4:junit4]   2> 10021 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.MMapDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/index4945549455tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@3c417f94),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 10022 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 10023 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 10024 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 10025 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 10025 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 10036 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 10056 T11 oass.SolrIndexSearcher.<init> Opening Searcher@31eb6efa main
[junit4:junit4]   2> 10056 T11 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 10057 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 10058 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 10059 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 10059 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 10060 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 10060 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 10061 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 10061 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 10062 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 10062 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 10063 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 10063 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 10080 T54 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@31eb6efa main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 10079 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 10080 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 10081 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 10082 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 10094 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 10096 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:13188
[junit4:junit4]   2> 10097 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 10098 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 10098 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 10136 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 10137 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 10137 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 10138 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 10138 T11 oasc.CoreContainer.<init> New CoreContainer 274270528
[junit4:junit4]   2> 10139 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 10140 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 10172 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 10191 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 10192 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 10193 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 10194 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 10241 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 10291 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 10293 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 10332 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 10926 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 10941 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 10945 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 10965 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10971 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10976 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10978 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10981 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard2/data/
[junit4:junit4]   2> 10982 T11 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 10983 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard2/data/index/
[junit4:junit4]   2> 10984 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard2/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 10985 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard2/data/index forceNew:false
[junit4:junit4]   2> 10989 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5b895852 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ae51802),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 10989 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 10991 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 10991 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 10992 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 10993 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 11006 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 11013 T11 oass.SolrIndexSearcher.<init> Opening Searcher@64f8145f main
[junit4:junit4]   2> 11014 T11 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 11015 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 11016 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 11016 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 11017 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 11017 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 11018 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 11018 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 11019 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 11019 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 11020 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 11020 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 11021 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 11034 T68 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@64f8145f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 11034 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 11035 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 11035 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 11036 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 11047 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 11050 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:65108
[junit4:junit4]   2> 11050 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 11051 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 11052 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 11093 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 11094 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 11094 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 11095 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 11095 T11 oasc.CoreContainer.<init> New CoreContainer 2108989886
[junit4:junit4]   2> 11096 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 11097 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 11129 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 11149 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 11150 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 11152 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 11153 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 11201 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 11258 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 11259 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 11293 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 11849 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 11865 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 11869 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 11889 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 11894 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 11900 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 11902 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 11904 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard3/data/
[junit4:junit4]   2> 11905 T11 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 11906 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard3/data/index/
[junit4:junit4]   2> 11907 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard3/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 11908 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1351879716743/shard3/data/index forceNew:false
[junit4:junit4]   2> 11911 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@26fb5989 lockFactory=org.apache.lucene.store.NativeFSLockFactory@128efd57),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 11912 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 11913 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 11914 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 11914 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 11915 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 11925 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 11931 T11 oass.SolrIndexSearcher.<init> Opening Searcher@5aa23fc8 main
[junit4:junit4]   2> 11931 T11 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 11932 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 11933 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 11934 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 11934 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 11935 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 11935 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 11936 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 11936 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 11937 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 11937 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 11937 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 11938 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 11951 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 11951 T82 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5aa23fc8 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 11952 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 11953 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 11954 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C0 name=collection1 org.apache.solr.core.SolrCore@281c7736
[junit4:junit4]   2> 13515 T22 C0 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@555a7085 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5dcd1c83; maxCacheMB=0.2607421875 maxMergeSizeMB=0.7861328125)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 13520 T22 C0 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 13579 T22 C0 oass.SolrIndexSearcher.<init> Opening Searcher@6201a5c realtime
[junit4:junit4]   2> 13580 T22 C0 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 13588 T22 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1417548639409012736)} 0 745
[junit4:junit4]   2> ASYNC  NEW_CORE C1 name=collection1 org.apache.solr.core.SolrCore@463fead8
[junit4:junit4]   2> 13792 T38 C1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@614c0482 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c7e3f6b),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 13793 T38 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 13795 T38 C1 oass.SolrIndexSearcher.<init> Opening Searcher@3316d58c realtime
[junit4:junit4]   2> 13796 T38 C1 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 13797 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1417548640092684288)} 0 9
[junit4:junit4]   2> ASYNC  NEW_CORE C2 name=collection1 org.apache.solr.core.SolrCore@2e5d21b2
[junit4:junit4]   2> 13808 T53 C2 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.MMapDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/index4945549455tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@3c417f94),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 13808 T53 C2 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 13810 T53 C2 oass.SolrIndexSearcher.<init> Opening Searcher@51c3358f realtime
[junit4:junit4]   2> 13810 T53 C2 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 13811 T53 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1417548640109461504)} 0 8
[junit4:junit4]   2> ASYNC  NEW_CORE C3 name=collection1 org.apache.solr.core.SolrCore@532ba0c2
[junit4:junit4]   2> 13820 T66 C3 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5b895852 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ae51802),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 13821 T66 C3 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 13822 T66 C3 oass.SolrIndexSearcher.<init> Opening Searcher@37f4ff60 realtime
[junit4:junit4]   2> 13823 T66 C3 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 13823 T66 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1417548640124141568)} 0 6
[junit4:junit4]   2> ASYNC  NEW_CORE C4 name=collection1 org.apache.solr.core.SolrCore@74a34961
[junit4:junit4]   2> 13835 T80 C4 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@26fb5989 lockFactory=org.apache.lucene.store.NativeFSLockFactory@128efd57),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 13836 T80 C4 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 13838 T80 C4 oass.SolrIndexSearcher.<init> Opening Searcher@84ffc19 realtime
[junit4:junit4]   2> 13839 T80 C4 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 13840 T80 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1417548640138821632)} 0 9
[junit4:junit4]   2> 13859 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16
[junit4:junit4]   2> 14406 T24 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1 (1417548640271990784)]} 0 461
[junit4:junit4]   2> 14414 T11 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:61902/solr/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4]   2> 14437 T25 C0 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 14555 T25 C0 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@555a7085 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5dcd1c83; maxCacheMB=0.2607421875 maxMergeSizeMB=0.7861328125)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@555a7085 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5dcd1c83; maxCacheMB=0.2607421875 maxMergeSizeMB=0.7861328125)),segFN=segments_2,generation=2,filenames=[_0_Lucene40_0.tim, _0.fnm, _0_Lucene40_0.frq, _0_nrm.cfe, segments_2, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene40_0.prx, _0.fdt, _0_Lucene40_0.tip]
[junit4:junit4]   2> 14555 T25 C0 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 14682 T25 C0 oass.SolrIndexSearcher.<init> Opening Searcher@2554373f main
[junit4:junit4]   2> 14683 T25 C0 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 14687 T25 C0 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 14694 T26 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2554373f main{StandardDirectoryReader(segments_2:3:nrt _0(4.1):C1)}
[junit4:junit4]   2> 14695 T25 C0 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&commit_end_point=true&expungeDeletes=false&wt=javabin&softCommit=false&version=2} {commit=} 0 258
[junit4:junit4]   2> 14840 T24 C0 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=1 status=0 QTime=118 
[junit4:junit4]   2> 14879 T22 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&commit_end_point=true&version=2} {add=[2 (1417548641231437824), 3 (1417548641234583552)]} 0 5
[junit4:junit4]   2> 14881 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&commit_end_point=true&version=2} {add=[2 (1417548641230389248), 3 (1417548641233534976), 4 (1417548641236680704)]} 0 9
[junit4:junit4]   2> 14883 T11 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:61902/solr/, StdNode: http://127.0.0.1:11368/solr/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4]   2> 14887 T24 C0 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 14887 T37 C1 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 14893 T37 C1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@614c0482 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c7e3f6b),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@614c0482 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c7e3f6b),segFN=segments_2,generation=2,filenames=[_0_Lucene40_0.tim, _0.fnm, _0_Lucene40_0.frq, _0_nrm.cfe, segments_2, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene40_0.prx, _0.fdt, _0_Lucene40_0.tip]
[junit4:junit4]   2> 14893 T37 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 14896 T37 C1 oass.SolrIndexSearcher.<init> Opening Searcher@d8009ee main
[junit4:junit4]   2> 14896 T37 C1 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 14897 T37 C1 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 14897 T40 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@d8009ee main{StandardDirectoryReader(segments_2:3:nrt _0(4.1):C3)}
[junit4:junit4]   2> 14899 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&commit_end_point=true&expungeDeletes=false&wt=javabin&softCommit=false&version=2} {commit=} 0 12
[junit4:junit4]   2> 14937 T24 C0 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@555a7085 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5dcd1c83; maxCacheMB=0.2607421875 maxMergeSizeMB=0.7861328125)),segFN=segments_2,generation=2,filenames=[_0_Lucene40_0.tim, _0.fnm, _0_Lucene40_0.frq, _0_nrm.cfe, segments_2, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene40_0.prx, _0.fdt, _0_Lucene40_0.tip]
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@555a7085 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5dcd1c83; maxCacheMB=0.2607421875 maxMergeSizeMB=0.7861328125)),segFN=segments_3,generation=3,filenames=[_1_Lucene40_0.tim, _1_nrm.cfs, _0_Lucene40_0.tim, _0.fnm, _1_Lucene40_0.frq, _0_nrm.cfs, _0_Lucene40_0.prx, _1.fnm, _1_Lucene40_0.prx, _1_Lucene40_0.tip, _1_nrm.cfe, _0_Lucene40_0.tip, _1.fdx, _0_Lucene40_0.frq, _1.fdt, _1.si, _0_nrm.cfe, _0.fdx, _0.si, segments_3, _0.fdt]
[junit4:junit4]   2> 14937 T24 C0 oasc.SolrDeletionPolicy.updateCommits newest commit = 3
[junit4:junit4]   2> 14943 T24 C0 oass.SolrIndexSearcher.<init> Opening Searcher@50df92ed main
[junit4:junit4]   2> 14944 T24 C0 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 14945 T26 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@50df92ed main{StandardDirectoryReader(segments_3:5:nrt _0(4.1):C1 _1(4.1):C2)}
[junit4:junit4]   2> 14945 T24 C0 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 14946 T24 C0 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&commit_end_point=true&expungeDeletes=false&wt=javabin&softCommit=false&version=2} {commit=} 0 59
[junit4:junit4]   2> 14950 T25 C0 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=3 status=0 QTime=1 
[junit4:junit4]   2> 14961 T38 C1 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=3 status=0 QTime=1 
[junit4:junit4]   2> 14970 T24 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&commit_end_point=true&version=2} {delete=[2 (-1417548641331052544)]} 0 1
[junit4:junit4]   2> 14971 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&commit_end_point=true&version=2} {delete=[2 (-1417548641332101120)]} 0 2
[junit4:junit4]   2> 14972 T11 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:61902/solr/, StdNode: http://127.0.0.1:11368/solr/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4]   2> 14976 T22 C0 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 14976 T37 C1 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 14991 T37 C1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@614c0482 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c7e3f6b),segFN=segments_2,generation=2,filenames=[_0_Lucene40_0.tim, _0.fnm, _0_Lucene40_0.frq, _0_nrm.cfe, segments_2, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene40_0.prx, _0.fdt, _0_Lucene40_0.tip]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@614c0482 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c7e3f6b),segFN=segments_3,generation=3,filenames=[_0_Lucene40_0.tim, _0.fnm, _0_Lucene40_0.frq, _0_nrm.cfe, _0_1.del, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene40_0.prx, _0.fdt, segments_3, _0_Lucene40_0.tip]
[junit4:junit4]   2> 14991 T22 C0 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@555a7085 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5dcd1c83; maxCacheMB=0.2607421875 maxMergeSizeMB=0.7861328125)),segFN=segments_3,generation=3,filenames=[_1_Lucene40_0.tim, _1_nrm.cfs, _0_Lucene40_0.tim, _0.fnm, _1_Lucene40_0.frq, _0_nrm.cfs, _0_Lucene40_0.prx, _1.fnm, _1_Lucene40_0.prx, _1_Lucene40_0.tip, _1_nrm.cfe, _0_Lucene40_0.tip, _1.fdx, _0_Lucene40_0.frq, _1.fdt, _1.si, _0_nrm.cfe, _0.fdx, _0.si, segments_3, _0.fdt]
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@555a7085 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5dcd1c83; maxCacheMB=0.2607421875 maxMergeSizeMB=0.7861328125)),segFN=segments_4,generation=4,filenames=[_1_Lucene40_0.tim, _1_nrm.cfs, _0_Lucene40_0.tim, _0.fnm, _1_Lucene40_0.frq, _0_nrm.cfs, _1_1.del, _0_Lucene40_0.prx, _1.fnm, _1_Lucene40_0.prx, _1_Lucene40_0.tip, _1_nrm.cfe, _0_Lucene40_0.tip, _1.fdx, _0_Lucene40_0.frq, _1.fdt, _1.si, _0_nrm.cfe, _0.fdx, _0.si, _0.fdt, segments_4]
[junit4:junit4]   2> 14991 T37 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 3
[junit4:junit4]   2> 14992 T22 C0 oasc.SolrDeletionPolicy.updateCommits newest commit = 4
[junit4:junit4]   2> 14993 T37 C1 oass.SolrIndexSearcher.<init> Opening Searcher@4dc679c8 main
[junit4:junit4]   2> 14993 T37 C1 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 14994 T40 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4dc679c8 main{StandardDirectoryReader(segments_3:4:nrt _0(4.1):C3/1)}
[junit4:junit4]   2> 14994 T37 C1 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 14995 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&commit_end_point=true&expungeDeletes=false&wt=javabin&softCommit=false&version=2} {commit=} 0 19
[junit4:junit4]   2> 14997 T22 C0 oass.SolrIndexSearcher.<init> Opening Searcher@2ee5d440 main
[junit4:junit4]   2> 14997 T22 C0 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 14998 T26 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2ee5d440 main{StandardDirectoryReader(segments_4:6:nrt _0(4.1):C1 _1(4.1):C2/1)}
[junit4:junit4]   2> 14998 T22 C0 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 14999 T22 C0 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&commit_end_point=true&expungeDeletes=false&wt=javabin&softCommit=false&version=2} {commit=} 0 23
[junit4:junit4]   2> 15003 T25 C0 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=2 status=0 QTime=1 
[junit4:junit4]   2> 15008 T38 C1 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=2 status=0 QTime=1 
[junit4:junit4]   2> 15012 T37 C1 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15399 T37 C1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@614c0482 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c7e3f6b),segFN=segments_3,generation=3,filenames=[_0_Lucene40_0.tim, _0.fnm, _0_Lucene40_0.frq, _0_nrm.cfe, _0_1.del, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene40_0.prx, _0.fdt, segments_3, _0_Lucene40_0.tip]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@614c0482 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c7e3f6b),segFN=segments_4,generation=4,filenames=[_1_Lucene40_0.tim, _1.fdx, _1_nrm.cfs, _1.fdt, _1.si, _1_Lucene40_0.frq, _1.fnm, _1_Lucene40_0.prx, _1_Lucene40_0.tip, _1_nrm.cfe, segments_4]
[junit4:junit4]   2> 15400 T37 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 4
[junit4:junit4]   2> 15405 T37 C1 oass.SolrIndexSearcher.<init> Opening Searcher@19b161e6 main
[junit4:junit4]   2> 15405 T37 C1 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 15406 T37 C1 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15406 T40 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@19b161e6 main{StandardDirectoryReader(segments_4:6:nrt _1(4.1):C2)}
[junit4:junit4]   2> 15408 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={optimize=true&maxSegments=1&waitSearcher=true&wt=javabin&version=2} {optimize=} 0 396
[junit4:junit4]   2> 15412 T51 C2 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15414 T51 C2 oass.SolrIndexSearcher.<init> Opening Searcher@393ddf54 main
[junit4:junit4]   2> 15414 T51 C2 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 15415 T54 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@393ddf54 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4]   2> 15415 T51 C2 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15416 T51 C2 UPDATE [collection1] webapp=/solr path=/update params={optimize=true&maxSegments=1&waitSearcher=true&wt=javabin&version=2} {optimize=} 0 4
[junit4:junit4]   2> 15421 T67 C3 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15422 T67 C3 oass.SolrIndexSearcher.<init> Opening Searcher@59d3b486 main
[junit4:junit4]   2> 15423 T67 C3 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 15423 T68 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@59d3b486 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4]   2> 15424 T67 C3 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15424 T67 C3 UPDATE [collection1] webapp=/solr path=/update params={optimize=true&maxSegments=1&waitSearcher=true&wt=javabin&version=2} {optimize=} 0 3
[junit4:junit4]   2> 15428 T80 C4 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15430 T80 C4 oass.SolrIndexSearcher.<init> Opening Searcher@1d2ab101 main
[junit4:junit4]   2> 15430 T80 C4 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 15431 T82 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1d2ab101 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4]   2> 15431 T80 C4 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15432 T80 C4 UPDATE [collection1] webapp=/solr path=/update params={optimize=true&maxSegments=1&waitSearcher=true&wt=javabin&version=2} {optimize=} 0 4
[junit4:junit4]   2> 15703 T78 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[6 (1417548641867923456), 8 (1417548641876312064), 10 (1417548641881554944), 14 (1417548642058764288), 15 (1417548642061910016), 17 (1417548642067152896), 19 (1417548642072395776), 21 (1417548642083930112), 22 (1417548642087075840), 24 (1417548642090221568)]} 0 221
[junit4:junit4]   2> 16053 T65 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[7 (1417548641870020608), 8 (1417548641879457792), 11 (1417548642067152896), 12 (1417548642072395777), 14 (1417548642078687233), 17 (1417548642091270144), 19 (1417548642096513024), 22 (1417548642433105920), 23 (1417548642455126017), 25 (1417548642463514624)]} 0 571
[junit4:junit4]   2> 16069 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[33 (1417548641876312065), 35 (1417548641904623616), 37 (1417548641907769344), 38 (1417548642061910016), 40 (1417548642069250048), 41 (1417548642071347201), 43 (1417548642075541504), 49 (1417548642078687232), 52 (1417548642089172993), 53 (1417548642101755904)]} 0 581
[junit4:junit4]   2> 16070 T98 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[66 (1417548641879457792), 68 (1417548642071347200), 69 (1417548642077638656), 70 (1417548642087075840), 71 (1417548642090221568), 72 (1417548642092318720), 73 (1417548642101755904), 75 (1417548642434154497), 76 (1417548642444640256), 78 (1417548642446737408)]} 0 578
[junit4:junit4]   2> 16073 T105 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[83 (1417548642068201472), 85 (1417548642075541505), 87 (1417548642082881536), 89 (1417548642086027264), 90 (1417548642089172993), 91 (1417548642093367296), 92 (1417548642103853056), 94 (1417548642434154496), 95 (1417548642437300224), 97 (1417548642443591680)]} 0 405
[junit4:junit4]   2> 16070 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[6 (1417548641871069184), 8 (1417548641876312064), 9 (1417548641879457792), 15 (1417548641881554944), 19 (1417548642070298624), 23 (1417548642076590081), 24 (1417548642087075840), 25 (1417548642091270144), 29 (1417548642093367296), 30 (1417548642433105920)]} 0 586
[junit4:junit4]   2> 16073 T52 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[5 (1417548641868972032), 6 (1417548641880506368), 9 (1417548642062958592), 14 (1417548642071347201), 15 (1417548642077638657), 18 (1417548642089172994), 20 (1417548642102804480), 21 (1417548642435203072), 22 (1417548642439397377), 23 (1417548642444640257)]} 0 591
[junit4:junit4]   2> 16071 T103 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[65 (1417548642064007168), 69 (1417548642075541504), 71 (1417548642080784384), 73 (1417548642088124417), 76 (1417548642091270144), 79 (1417548642101755904), 80 (1417548642103853056), 82 (1417548642108047360), 83 (1417548642436251648), 84 (1417548642460368897)]} 0 403
[junit4:junit4]   2> 16082 T81 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[40 (1417548641880506368), 41 (1417548642071347200), 46 (1417548642084978688), 48 (1417548642092318720), 51 (1417548642096513025), 52 (1417548642106998784), 53 (1417548642432057344), 55 (1417548642460368896), 59 (1417548642462466048), 60 (1417548642486583296)]} 0 590
[junit4:junit4]   2> 16085 T51 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[24 (1417548641872117760), 27 (1417548642060861440), 28 (1417548642075541504), 34 (1417548642088124416), 38 (1417548642091270144), 39 (1417548642105950208), 41 (1417548642440445952), 43 (1417548642444640258), 44 (1417548642486583296), 46 (1417548642491826176)]} 0 601
[junit4:junit4]   2> 16098 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[59 (1417548641877360640), 61 (1417548642058764288), 62 (1417548642068201472), 63 (1417548642071347200), 64 (1417548642076590080), 65 (1417548642089172992), 69 (1417548642433105921), 73 (1417548642485534722), 74 (1417548642500214785), 75 (1417548642504409088)]} 0 609
[junit4:junit4]   2> 16100 T80 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[26 (1417548641872117760), 27 (1417548642067152897), 29 (1417548642088124416), 31 (1417548642096513024), 33 (1417548642427863040), 35 (1417548642438348800), 36 (1417548642446737408), 37 (1417548642451980288), 38 (1417548642490777602), 39 (1417548642500214784)]} 0 616
[junit4:junit4]   2> 16101 T67 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[49 (1417548641879457793), 52 (1417548642071347200), 55 (1417548642078687232), 56 (1417548642086027264), 59 (1417548642091270145), 60 (1417548642096513025), 63 (1417548642103853056), 65 (1417548642431008768), 66 (1417548642441494528), 69 (1417548642443591680)]} 0 609
[junit4:junit4]   2> 16107 T118 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[116 (1417548642097561600), 117 (1417548642437300225), 118 (1417548642485534721), 119 (1417548642491826178), 123 (1417548642499166208), 124 (1417548642501263361), 125 (1417548642503360512), 126 (1417548642505457665), 128 (1417548642519089152), 131 (1417548642521186304)]} 0 411
[junit4:junit4]   2> 16107 T116 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[113 (1417548642094415872), 115 (1417548642449883136), 116 (1417548642454077440), 117 (1417548642463514624), 119 (1417548642488680448), 120 (1417548642490777601), 122 (1417548642501263360), 124 (1417548642503360512), 125 (1417548642506506241), 126 (1417548642516992000)]} 0 411
[junit4:junit4]   2> 16110 T53 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[50 (1417548641877360640), 51 (1417548642069250048), 53 (1417548642076590080), 54 (1417548642082881537), 55 (1417548642092318721), 57 (1417548642097561600), 59 (1417548642431008768), 60 (1417548642448834560), 62 (1417548642511749120), 64 (1417548642514894848)]} 0 621
[junit4:junit4]   2> 16112 T66 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[26 (1417548641877360640), 27 (1417548641908817920), 28 (1417548642069250048), 33 (1417548642075541504), 36 (1417548642082881537), 37 (1417548642094415872), 40 (1417548642427863040), 43 (1417548642438348800), 45 (1417548642485534720), 48 (1417548642514894848)]} 0 622
[junit4:junit4]   2> 16115 T113 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[97 (1417548642083930112), 98 (1417548642096513024), 100 (1417548642431008768), 103 (1417548642489729024), 105 (1417548642491826177), 111 (1417548642501263360), 112 (1417548642509651969), 113 (1417548642514894849), 114 (1417548642519089153), 115 (1417548642522234881)]} 0 429
[junit4:junit4]   2> 16116 T115 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[98 (1417548642089172992), 102 (1417548642094415872), 103 (1417548642105950209), 106 (1417548642440445953), 108 (1417548642497069057), 111 (1417548642499166209), 112 (1417548642508603393), 115 (1417548642515943424), 116 (1417548642523283456), 117 (1417548642527477760)]} 0 424
[junit4:junit4]   2> 16117 T125 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[136 (1417548642435203072), 137 (1417548642442543104), 138 (1417548642445688832), 140 (1417548642448834560), 142 (1417548642451980288), 143 (1417548642455126016), 144 (1417548642460368896), 145 (1417548642465611776), 146 (1417548642468757504), 147 (1417548642527477760)]} 0 95
[junit4:junit4]   2> 16123 T102 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[77 (1417548642067152896), 79 (1417548642073444352), 80 (1417548642080784384), 81 (1417548642091270145), 82 (1417548642435203072), 83 (1417548642485534723), 84 (1417548642491826176), 89 (1417548642502311937), 90 (1417548642505457666), 96 (1417548642514894848)]} 0 452
[junit4:junit4]   2> 16136 T81 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[212 (1417548642520137728), 213 (1417548642523283456), 216 (1417548642525380608), 218 (1417548642527477761), 222 (1417548642532720641), 223 (1417548642539012096), 224 (1417548642541109249), 226 (1417548642548449280), 227 (1417548642550546432), 228 (1417548642552643584)]} 0 34
[junit4:junit4]   2> 16137 T127 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[118 (1417548642432057344), 120 (1417548642439397376), 126 (1417548642487631872), 127 (1417548642499166208), 130 (1417548642501263361), 131 (1417548642503360513), 132 (1417548642506506240), 134 (1417548642508603392), 136 (1417548642527477761), 137 (1417548642551595009)]} 0 119
[junit4:junit4]   2> 16138 T129 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[138 (1417548642459320320), 139 (1417548642497069058), 141 (1417548642503360512), 142 (1417548642512797696), 143 (1417548642519089152), 144 (1417548642522234880), 150 (1417548642524332032), 152 (1417548642531672064), 157 (1417548642549497856), 160 (1417548642553692161)]} 0 96
[junit4:junit4]   2> 16140 T133 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[155 (1417548642449883136), 158 (1417548642490777600), 159 (1417548642498117632), 160 (1417548642502311936), 161 (1417548642507554816), 162 (1417548642532720640), 164 (1417548642540060672), 165 (1417548642544254976), 166 (1417548642550546432), 168 (1417548642552643585)]} 0 104
[junit4:junit4]   2> 16144 T111 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[88 (1417548642082881536), 90 (1417548642095464448), 93 (1417548642434154496), 95 (1417548642484486145), 101 (1417548642491826176), 102 (1417548642502311936), 103 (1417548642504409089), 104 (1417548642527477762), 109 (1417548642541109248), 110 (1417548642543206400)]} 0 459
[junit4:junit4]   2> 16146 T148 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[152 (1417548642465611776), 153 (1417548642491826177), 154 (1417548642506506240), 158 (1417548642510700544), 159 (1417548642532720642), 162 (1417548642540060672), 171 (1417548642544254976), 172 (1417548642555789312), 173 (1417548642558935040), 174 (1417548642561032193)]} 0 95
[junit4:junit4]   2> 16147 T142 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[196 (1417548642501263360), 197 (1417548642537963521), 199 (1417548642542157825), 200 (1417548642548449280), 205 (1417548642551595008), 209 (1417548642553692160), 212 (1417548642555789312), 213 (1417548642557886465), 214 (1417548642562080768), 216 (1417548642564177921)]} 0 63
[junit4:junit4]   2> 16148 T143 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[170 (1417548642488680448), 171 (1417548642504409090), 172 (1417548642509651968), 173 (1417548642513846273), 175 (1417548642522234880), 176 (1417548642526429184), 179 (1417548642541109248), 180 (1417548642555789312), 183 (1417548642559983617), 185 (1417548642563129344)]} 0 76
[junit4:junit4]   2> 16154 T112 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[100 (1417548642088124416), 101 (1417548642097561600), 103 (1417548642427863041), 105 (1417548642431008769), 106 (1417548642439397376), 107 (1417548642441494529), 110 (1417548642456174592), 111 (1417548642524332034), 114 (1417548642529574912), 116 (1417548642540060672)]} 0 463
[junit4:junit4]   2> 16155 T158 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[270 (1417548642514894850), 271 (1417548642532720641), 272 (1417548642540060673), 274 (1417548642548449282), 275 (1417548642552643584), 276 (1417548642555789313), 280 (1417548642558935041), 283 (1417548642563129345), 284 (1417548642567323650), 285 (1417548642570469378)]} 0 57
[junit4:junit4]   2> 16155 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[210 (1417548642505457664), 211 (1417548642506506240), 212 (1417548642512797696), 214 (1417548642520137728), 216 (1417548642523283456), 224 (1417548642525380608), 227 (1417548642527477760), 228 (1417548642535866368), 229 (1417548642544254977), 230 (1417548642561032192)]} 0 67
[junit4:junit4]   2> 16157 T128 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[164 (1417548642485534720), 172 (1417548642504409088), 174 (1417548642515943425), 175 (1417548642519089153), 177 (1417548642533769216), 179 (1417548642555789313), 180 (1417548642558935043), 181 (1417548642564177920), 182 (1417548642569420800), 185 (1417548642571517953)]} 0 88
[junit4:junit4]   2> 16162 T103 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[266 (1417548642535866368), 267 (1417548642537963520), 269 (1417548642559983616), 270 (1417548642564177920), 271 (1417548642566275072), 272 (1417548642571517953), 273 (1417548642573615104), 275 (1417548642576760832), 279 (1417548642578857984), 284 (1417548642580955136)]} 0 45
[junit4:junit4]   2> 16164 T119 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[132 (1417548642437300224), 134 (1417548642485534720), 135 (1417548642500214784), 139 (1417548642504409089), 140 (1417548642508603392), 141 (1417548642511749120), 145 (1417548642516992000), 151 (1417548642534817792), 152 (1417548642567323648), 154 (1417548642573615104)]} 0 140
[junit4:junit4]   2> 16167 T121 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[117 (1417548642100707329), 119 (1417548642439397377), 121 (1417548642443591681), 122 (1417548642521186304), 123 (1417548642525380608), 125 (1417548642531672064), 130 (1417548642541109248), 131 (1417548642567323648), 132 (1417548642570469376), 134 (1417548642579906560)]} 0 465
[junit4:junit4]   2> 16171 T65 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[218 (1417548642509651968), 219 (1417548642521186305), 220 (1417548642524332032), 222 (1417548642527477762), 223 (1417548642533769216), 224 (1417548642549497857), 231 (1417548642554740736), 232 (1417548642557886464), 233 (1417548642580955136), 237 (1417548642586198016)]} 0 78
[junit4:junit4]   2> 16171 T116 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[247 (1417548642531672064), 249 (1417548642562080768), 250 (1417548642572566529), 251 (1417548642575712256), 252 (1417548642577809408), 254 (1417548642578857986), 259 (1417548642580955137), 260 (1417548642584100865), 262 (1417548642587246592), 263 (1417548642590392320)]} 0 57
[junit4:junit4]   2> 16171 T144 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[189 (1417548642513846272), 194 (1417548642534817793), 197 (1417548642548449281), 198 (1417548642554740736), 199 (1417548642558935040), 205 (1417548642567323649), 206 (1417548642572566528), 207 (1417548642575712256), 208 (1417548642580955136), 209 (1417548642587246592)]} 0 75
[junit4:junit4]   2> 16174 T146 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[221 (1417548642559983616), 224 (1417548642565226498), 225 (1417548642572566528), 226 (1417548642575712256), 228 (1417548642577809408), 229 (1417548642579906560), 230 (1417548642583052289), 233 (1417548642586198019), 234 (1417548642588295168), 236 (1417548642591440896)]} 0 50
[junit4:junit4]   2> 16171 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[232 (1417548642533769216), 236 (1417548642536914944), 238 (1417548642548449280), 240 (1417548642551595008), 241 (1417548642555789314), 242 (1417548642559983618), 243 (1417548642565226496), 244 (1417548642568372224), 245 (1417548642570469376), 247 (1417548642589343744)]} 0 66
[junit4:junit4]   2> 16174 T105 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[238 (1417548642542157824), 239 (1417548642552643584), 245 (1417548642558935042), 246 (1417548642565226497), 247 (1417548642568372224), 249 (1417548642573615104), 253 (1417548642583052288), 254 (1417548642586198018), 255 (1417548642589343744), 256 (1417548642592489472)]} 0 50
[junit4:junit4]   2> 16177 T98 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[257 (1417548642531672065), 260 (1417548642548449280), 262 (1417548642566275072), 264 (1417548642571517952), 268 (1417548642579906561), 274 (1417548642584100864), 276 (1417548642587246592), 279 (1417548642590392320), 280 (1417548642592489473), 281 (1417548642594586624)]} 0 63
[junit4:junit4]   2> 16173 T51 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[282 (1417548642541109248), 283 (1417548642551595008), 285 (1417548642555789312), 288 (1417548642558935041), 289 (1417548642565226496), 291 (1417548642569420801), 293 (1417548642573615105), 295 (1417548642582003712), 297 (1417548642586198017), 298 (1417548642589343746)]} 0 51
[junit4:junit4]   2> 16178 T66 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[276 (1417548642532720640), 278 (1417548642542157824), 279 (1417548642549497856), 281 (1417548642551595009), 285 (1417548642572566528), 288 (1417548642577809408), 289 (1417548642584100864), 290 (1417548642588295168), 291 (1417548642593538048), 293 (1417548642595635200)]} 0 64
[junit4:junit4]   2> 16181 T52 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[205 (1417548642542157825), 206 (1417548642550546432), 209 (1417548642553692160), 210 (1417548642580955136), 211 (1417548642586198016), 212 (1417548642589343745), 214 (1417548642593538048), 217 (1417548642595635200), 218 (1417548642598780928), 219 (1417548642599829504)]} 0 57
[junit4:junit4]   2> 16181 T80 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[231 (1417548642533769216), 232 (1417548642561032192), 233 (1417548642569420800), 234 (1417548642576760833), 235 (1417548642582003712), 236 (1417548642584100864), 237 (1417548642586198017), 238 (1417548642588295168), 239 (1417548642593538048), 241 (1417548642595635200)]} 0 73
[junit4:junit4]   2> 16181 T67 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[240 (1417548642521186306), 242 (1417548642524332033), 243 (1417548642541109249), 245 (1417548642561032193), 247 (1417548642574663680), 250 (1417548642587246592), 253 (1417548642590392320), 254 (1417548642593538049), 255 (1417548642596683776), 256 (1417548642598780928)]} 0 77
[junit4:junit4]   2> 16181 T118 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[286 (1417548642571517952), 291 (1417548642576760832), 292 (1417548642580955137), 293 (1417548642586198016), 294 (1417548642589343745), 295 (1417548642591440896), 296 (1417548642593538048), 298 (1417548642595635200), 306 (1417548642597732352), 307 (1417548642600878080)]} 0 29
[junit4:junit4]   2> 16182 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[249 (1417548642513846274), 253 (1417548642537963520), 256 (1417548642559983616), 260 (1417548642564177920), 262 (1417548642570469377), 263 (1417548642578857984), 265 (1417548642584100864), 266 (1417548642588295168), 268 (1417548642594586624), 269 (1417548642598780928)]} 0 85
[junit4:junit4]   2> 16187 T111 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[364 (1417548642574663680), 367 (1417548642578857985), 369 (1417548642582003713), 371 (1417548642586198016), 372 (1417548642590392321), 376 (1417548642599829504), 377 (1417548642602975232), 378 (1417548642604023808), 379 (1417548642605072384), 383 (1417548642606120960)]} 0 32
[junit4:junit4]   2> 16502 T78 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[128 (1417548642435203072), 133 (1417548642441494528), 135 (1417548642484486144), 136 (1417548642499166208), 139 (1417548642504409088), 141 (1417548642507554816), 143 (1417548642510700545), 144 (1417548642513846272), 145 (1417548642518040576), 148 (1417548642533769217)]} 0 480
[junit4:junit4]   2> 16502 T104 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[71 (1417548642072395776), 72 (1417548642082881536), 74 (1417548642089172992), 76 (1417548642094415873), 80 (1417548642100707328), 83 (1417548642436251648), 90 (1417548642487631872), 91 (1417548642523283457), 94 (1417548642525380609), 95 (1417548642530623489)]} 0 827
[junit4:junit4]   2> 16503 T145 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[187 (1417548642497069056), 189 (1417548642501263360), 190 (1417548642504409089), 191 (1417548642510700544), 193 (1417548642514894849), 197 (1417548642516992000), 199 (1417548642548449281), 200 (1417548642558935040), 202 (1417548642561032192), 204 (1417548642938519552)]} 0 431
[junit4:junit4]   2> 16504 T134 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[149 (1417548642459320320), 150 (1417548642518040576), 151 (1417548642520137728), 153 (1417548642523283456), 156 (1417548642527477761), 157 (1417548642533769217), 158 (1417548642561032192), 159 (1417548642570469377), 160 (1417548642576760833), 165 (1417548642583052289)]} 0 462
[junit4:junit4]   2> 16505 T161 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[257 (1417548642530623488), 260 (1417548642533769218), 262 (1417548642559983616), 263 (1417548642564177920), 265 (1417548642568372224), 266 (1417548642576760832), 269 (1417548642578857984), 272 (1417548642583052288), 274 (1417548642938519552), 275 (1417548642939568129)]} 0 393
[junit4:junit4]   2> 16505 T149 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[176 (1417548642490777600), 178 (1417548642507554817), 179 (1417548642527477763), 180 (1417548642532720640), 183 (1417548642537963521), 185 (1417548642540060673), 187 (1417548642559983617), 191 (1417548642564177921), 192 (1417548642566275073), 194 (1417548642571517952)]} 0 430
[junit4:junit4]   2> 16511 T124 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[196 (1417548642488680449), 197 (1417548642515943424), 198 (1417548642519089152), 200 (1417548642521186304), 201 (1417548642527477760), 204 (1417548642531672065), 205 (1417548642567323648), 206 (1417548642572566528), 207 (1417548642943762432), 209 (1417548642945859584)]} 0 439
[junit4:junit4]   2> 16517 T53 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[341 (1417548642938519553), 343 (1417548642940616704), 345 (1417548642941665280), 346 (1417548642942713856), 348 (1417548642944811008), 349 (1417548642945859584), 350 (1417548642946908161), 352 (1417548642950053888), 354 (1417548642951102464), 355 (1417548642953199616)]} 0 15
[junit4:junit4]   2> 16519 T133 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[328 (1417548642942713856), 329 (1417548642943762433), 332 (1417548642944811009), 333 (1417548642945859585), 336 (1417548642947956737), 339 (1417548642949005313), 343 (1417548642950053889), 344 (1417548642951102465), 345 (1417548642953199616), 346 (1417548642954248193)]} 0 14
[junit4:junit4]   2> 16520 T113 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[348 (1417548642941665280), 350 (1417548642943762432), 351 (1417548642944811008), 352 (1417548642945859584), 353 (1417548642947956736), 360 (1417548642949005312), 361 (1417548642950053888), 362 (1417548642951102464), 366 (1417548642953199617), 368 (1417548642954248194)]} 0 15
[junit4:junit4]   2> 16520 T141 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[167 (1417548642489729024), 172 (1417548642514894849), 173 (1417548642537963520), 181 (1417548642940616704), 183 (1417548642943762432), 187 (1417548642945859584), 188 (1417548642947956737), 191 (1417548642949005314), 193 (1417548642951102466), 195 (1417548642954248192)]} 0 447
[junit4:junit4]   2> 16523 T81 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[287 (1417548642939568128), 289 (1417548642941665280), 291 (1417548642943762433), 296 (1417548642945859585), 299 (1417548642947956736), 300 (1417548642949005312), 301 (1417548642951102464), 304 (1417548642953199616), 305 (1417548642956345344), 306 (1417548642958442496)]} 0 20
[junit4:junit4]   2> 16525 T125 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[343 (1417548642943762433), 344 (1417548642944811009), 346 (1417548642945859585), 348 (1417548642947956738), 349 (1417548642949005313), 350 (1417548642950053890), 352 (1417548642952151040), 353 (1417548642953199618), 356 (1417548642954248193), 364 (1417548642957393920)]} 0 18
[junit4:junit4]   2> 16528 T115 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[356 (1417548642941665281), 357 (1417548642943762432), 359 (1417548642944811009), 363 (1417548642945859585), 366 (1417548642949005312), 367 (1417548642950053889), 368 (1417548642952151040), 371 (1417548642956345344), 373 (1417548642958442496), 376 (1417548642962636800)]} 0 23
[junit4:junit4]   2> 16530 T112 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[320 (1417548642944811008), 322 (1417548642946908161), 326 (1417548642949005312), 328 (1417548642951102464), 329 (1417548642953199617), 332 (1417548642955296769), 333 (1417548642957393922), 336 (1417548642960539649), 339 (1417548642963685377), 341 (1417548642965782528)]} 0 22
[junit4:junit4]   2> 16531 T159 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[295 (1417548642939568128), 296 (1417548642940616705), 298 (1417548642942713856), 301 (1417548642947956736), 302 (1417548642950053889), 304 (1417548642953199616), 308 (1417548642956345344), 315 (1417548642959491073), 317 (1417548642963685376), 318 (1417548642967879681)]} 0 29
[junit4:junit4]   2> 16536 T118 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[403 (1417548642957393921), 405 (1417548642960539648), 407 (1417548642962636800), 411 (1417548642963685377), 412 (1417548642965782528), 413 (1417548642966831104), 414 (1417548642967879682), 417 (1417548642969976833), 418 (1417548642971025410), 419 (1417548642973122560)]} 0 16
[junit4:junit4]   2> 16538 T158 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[369 (1417548642954248192), 372 (1417548642955296768), 373 (1417548642959491072), 374 (1417548642961588225), 378 (1417548642963685376), 379 (1417548642965782530), 382 (1417548642967879681), 384 (1417548642969976834), 385 (1417548642972073985), 387 (1417548642974171136)]} 0 22
[junit4:junit4]   2> 16540 T128 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[300 (1417548642944811010), 302 (1417548642946908160), 303 (1417548642949005313), 304 (1417548642951102465), 306 (1417548642954248192), 307 (1417548642960539649), 309 (1417548642963685376), 311 (1417548642967879682), 315 (1417548642971025409), 319 (1417548642976268288)]} 0 32
[junit4:junit4]   2> 16592 T65 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[389 (1417548642946908160), 390 (1417548642951102465), 393 (1417548642955296768), 399 (1417548642957393921), 400 (1417548642960539648), 401 (1417548642965782529), 402 (1417548642968928256), 404 (1417548642973122561), 405 (1417548642975219713), 406 (1417548642977316865)]} 0 83
[junit4:junit4]   2> 16593 T146 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[392 (1417548642957393920), 393 (1417548642960539648), 395 (1417548642961588224), 396 (1417548642965782528), 399 (1417548642967879681), 401 (1417548642968928256), 403 (1417548642973122560), 405 (1417548642976268289), 407 (1417548642977316864), 408 (1417548642979414016)]} 0 74
[junit4:junit4]   2> 16595 T66 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[365 (1417548642950053888), 369 (1417548642952151041), 372 (1417548642955296770), 373 (1417548642964733952), 374 (1417548642967879680), 375 (1417548642971025409), 376 (1417548642975219712), 377 (1417548642977316864), 381 (1417548642978365440), 388 (1417548643033939968)]} 0 82
[junit4:junit4]   2> 16596 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[308 (1417548642962636801), 309 (1417548642963685378), 313 (1417548642965782531), 314 (1417548642967879680), 315 (1417548642971025409), 319 (1417548642973122561), 321 (1417548642975219712), 325 (1417548642976268289), 326 (1417548642979414017), 327 (1417548643033939970)]} 0 71
[junit4:junit4]   2> 16599 T144 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[494 (1417548642968928257), 496 (1417548642972073984), 497 (1417548642973122562), 498 (1417548642976268288), 501 (1417548642977316865), 503 (1417548643032891392), 506 (1417548643033939969), 507 (1417548643036037121), 508 (1417548643038134272), 509 (1417548643039182849)]} 0 68
[junit4:junit4]   2> 16599 T119 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[450 (1417548642956345344), 451 (1417548642958442496), 453 (1417548642961588224), 454 (1417548642964733952), 456 (1417548642969976832), 460 (1417548642972073986), 461 (1417548642975219713), 466 (1417548642979414016), 470 (1417548643034988544), 476 (1417548643037085696)]} 0 81
[junit4:junit4]   2> 16599 T116 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[427 (1417548642955296768), 428 (1417548642957393920), 429 (1417548642959491073), 430 (1417548642964733952), 433 (1417548642965782528), 437 (1417548642969976832), 439 (1417548642972073984), 441 (1417548642974171138), 444 (1417548642976268288), 445 (1417548642979414016)]} 0 81
[junit4:junit4]   2> 16609 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[420 (1417548642957393920), 421 (1417548642960539649), 422 (1417548642965782529), 423 (1417548642968928256), 426 (1417548642971025408), 429 (1417548642974171137), 434 (1417548642977316864), 435 (1417548643033939968), 439 (1417548643036037120), 443 (1417548643038134273)]} 0 89
[junit4:junit4]   2> 16618 T98 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[450 (1417548642960539650), 451 (1417548642963685377), 452 (1417548642967879680), 454 (1417548642971025408), 455 (1417548642975219712), 456 (1417548642978365440), 458 (1417548643042328576), 461 (1417548643044425728), 463 (1417548643045474304), 465 (1417548643057008642)]} 0 94
[junit4:junit4]   2> 16623 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[388 (1417548643040231424), 390 (1417548643045474305), 392 (1417548643050717185), 394 (1417548643052814336), 395 (1417548643053862912), 396 (1417548643054911489), 398 (1417548643057008641), 399 (1417548643059105792), 400 (1417548643061202944), 402 (1417548643063300096)]} 0 25
[junit4:junit4]   2> 16623 T80 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[406 (1417548642959491072), 410 (1417548642967879681), 412 (1417548642969976833), 414 (1417548642974171136), 416 (1417548642975219712), 417 (1417548642977316865), 419 (1417548642979414017), 421 (1417548643043377152), 424 (1417548643053862912), 425 (1417548643062251521)]} 0 101
[junit4:junit4]   2> 16633 T51 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[509 (1417548643034988544), 510 (1417548643036037120), 511 (1417548643038134272), 516 (1417548643039182849), 517 (1417548643040231424), 518 (1417548643041280001), 521 (1417548643043377152), 522 (1417548643044425729), 523 (1417548643059105792), 526 (1417548643066445824)]} 0 39
[junit4:junit4]   2> 16637 T104 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[453 (1417548642956345345), 454 (1417548642959491072), 461 (1417548642961588224), 462 (1417548642964733953), 463 (1417548642967879682), 468 (1417548642971025408), 469 (1417548642973122560), 470 (1417548643050717184), 472 (1417548643072737280), 477 (1417548643073785857)]} 0 116
[junit4:junit4]   2> 16647 T102 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[510 (1417548643037085697), 515 (1417548643039182848), 520 (1417548643040231425), 521 (1417548643041280001), 523 (1417548643043377152), 527 (1417548643044425729), 529 (1417548643050717184), 530 (1417548643052814337), 534 (1417548643054911488), 536 (1417548643059105793)]} 0 51
[junit4:junit4]   2> 16647 T158 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[583 (1417548643060154369), 586 (1417548643065397248), 588 (1417548643072737280), 590 (1417548643073785857), 591 (1417548643076931584), 593 (1417548643084271616), 594 (1417548643085320193), 596 (1417548643086368769), 597 (1417548643088465920), 600 (1417548643089514496)]} 0 33
[junit4:junit4]   2> 16648 T143 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[478 (1417548643041280000), 480 (1417548643047571456), 481 (1417548643051765760), 482 (1417548643052814338), 483 (1417548643053862914), 486 (1417548643055960064), 487 (1417548643057008642), 488 (1417548643058057216), 489 (1417548643061202945), 493 (1417548643084271618)]} 0 48
[junit4:junit4]   2> 16649 T113 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[537 (1417548643044425728), 541 (1417548643045474304), 542 (1417548643057008640), 545 (1417548643059105794), 548 (1417548643084271617), 549 (1417548643085320194), 551 (1417548643087417344), 554 (1417548643088465922), 558 (1417548643089514498), 562 (1417548643091611648)]} 0 47
[junit4:junit4]   2> 16652 T65 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[421 (1417548643036037120), 425 (1417548643038134272), 426 (1417548643042328576), 428 (1417548643044425729), 429 (1417548643058057216), 431 (1417548643073785859), 432 (1417548643082174465), 433 (1417548643085320194), 435 (1417548643087417346), 436 (1417548643089514498)]} 0 58
[junit4:junit4]   2> 16653 T160 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[407 (1417548643034988544), 409 (1417548643037085696), 410 (1417548643039182849), 411 (1417548643049668608), 412 (1417548643054911489), 414 (1417548643066445824), 416 (1417548643076931584), 418 (1417548643083223040), 419 (1417548643092660224), 420 (1417548643095805952)]} 0 59
[junit4:junit4]   2> 16653 T118 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[564 (1417548643049668608), 569 (1417548643053862913), 571 (1417548643055960065), 573 (1417548643060154368), 574 (1417548643066445824), 575 (1417548643073785856), 576 (1417548643076931585), 577 (1417548643092660224), 581 (1417548643093708800), 582 (1417548643095805952)]} 0 45
[junit4:junit4]   2> 16656 T124 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[446 (1417548643040231425), 449 (1417548643047571459), 450 (1417548643052814339), 453 (1417548643055960064), 455 (1417548643065397250), 456 (1417548643077980160), 457 (1417548643082174464), 459 (1417548643083223041), 460 (1417548643093708802), 461 (1417548643094757377)]} 0 57
[junit4:junit4]   2> 16661 T98 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[591 (1417548643064348672), 593 (1417548643081125888), 594 (1417548643083223041), 598 (1417548643085320194), 604 (1417548643087417347), 605 (1417548643090563072), 606 (1417548643091611649), 608 (1417548643095805954), 609 (1417548643097903104), 610 (1417548643101048832)]} 0 39
[junit4:junit4]   2> 16658 T129 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[320 (1417548643041280003), 323 (1417548643051765761), 326 (1417548643077980160), 328 (1417548643085320192), 331 (1417548643087417344), 332 (1417548643089514496), 335 (1417548643090563074), 336 (1417548643095805952), 339 (1417548643097903105), 340 (1417548643100000256)]} 0 58
[junit4:junit4]   2> 16663 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[603 (1417548643065397249), 604 (1417548643085320192), 605 (1417548643086368768), 606 (1417548643088465921), 608 (1417548643089514497), 609 (1417548643094757376), 610 (1417548643098951680), 611 (1417548643100000256), 613 (1417548643102097408), 617 (1417548643106291712)]} 0 40
[junit4:junit4]   2> 16664 T105 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[468 (1417548643057008641), 470 (1417548643079028738), 472 (1417548643086368769), 473 (1417548643088465921), 477 (1417548643092660225), 480 (1417548643095805953), 483 (1417548643097903106), 484 (1417548643100000257), 485 (1417548643105243136), 487 (1417548643107340288)]} 0 49
[junit4:junit4]   2> 16666 T149 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[385 (1417548643040231424), 388 (1417548643057008640), 391 (1417548643063300096), 392 (1417548643074834432), 396 (1417548643080077312), 398 (1417548643084271617), 399 (1417548643086368770), 401 (1417548643093708800), 404 (1417548643094757376), 405 (1417548643098951680)]} 0 67
[junit4:junit4]   2> 16666 T52 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[377 (1417548643039182848), 378 (1417548643047571458), 379 (1417548643051765762), 382 (1417548643053862912), 383 (1417548643054911488), 384 (1417548643079028736), 386 (1417548643081125889), 387 (1417548643083223040), 388 (1417548643106291712), 389 (1417548643108388864)]} 0 69
[junit4:junit4]   2> 16667 T165 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[511 (1417548643052814337), 512 (1417548643063300097), 513 (1417548643072737281), 519 (1417548643084271618), 522 (1417548643086368769), 523 (1417548643088465920), 524 (1417548643093708803), 528 (1417548643097903106), 531 (1417548643106291712), 532 (1417548643108388864)]} 0 56
[junit4:junit4]   2> 16657 T81 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[308 (1417548643047571456), 309 (1417548643053862913), 311 (1417548643061202944), 316 (1417548643072737282), 317 (1417548643074834433), 320 (1417548643081125888), 325 (1417548643082174466), 328 (1417548643086368768), 332 (1417548643095805952), 335 (1417548643096854528)]} 0 52
[junit4:junit4]   2> 16678 T128 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[554 (1417548643049668608), 555 (1417548643055960064), 556 (1417548643058057218), 559 (1417548643075883008), 565 (1417548643085320193), 566 (1417548643090563073), 567 (1417548643096854528), 569 (1417548643101048833), 570 (1417548643110486016), 574 (1417548643111534592)]} 0 70
[junit4:junit4]   2> 16679 T148 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[342 (1417548643034988544), 343 (1417548643051765760), 344 (1417548643054911489), 347 (1417548643062251520), 348 (1417548643064348672), 349 (1417548643080077316), 350 (1417548643082174468), 352 (1417548643093708801), 361 (1417548643097903104), 363 (1417548643102097409)]} 0 85
[junit4:junit4]   2> 16679 T121 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[540 (1417548643040231424), 541 (1417548643051765760), 543 (1417548643061202945), 544 (1417548643084271617), 546 (1417548643086368768), 547 (1417548643089514497), 548 (1417548643100000259), 552 (1417548643108388864), 555 (1417548643110486016), 557 (1417548643123068928)]} 0 80
[junit4:junit4]   2> 16657 T111 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[462 (1417548642960539648), 463 (1417548642962636800), 465 (1417548642967879680), 466 (1417548642972073985), 468 (1417548642974171137), 469 (1417548642977316864), 470 (1417548643032891392), 471 (1417548643050717185), 473 (1417548643084271616), 474 (1417548643093708804)]} 0 134
[junit4:junit4]   2> 16679 T159 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[608 (1417548643075883008), 609 (1417548643077980160), 611 (1417548643087417344), 612 (1417548643090563072), 614 (1417548643093708800), 617 (1417548643096854529), 618 (1417548643098951681), 619 (1417548643102097408), 625 (1417548643107340288), 626 (1417548643111534593)]} 0 46
[junit4:junit4]   2> 16668 T116 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[554 (1417548643050717184), 555 (1417548643058057217), 557 (1417548643060154368), 558 (1417548643065397249), 559 (1417548643074834434), 561 (1417548643080077313), 569 (1417548643082174467), 575 (1417548643088465921), 577 (1417548643105243136), 578 (1417548643110486017)]} 0 62
[junit4:junit4]   2> 16668 T53 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[432 (1417548643041280002), 433 (1417548643053862913), 436 (1417548643057008640), 437 (1417548643086368768), 439 (1417548643089514497), 440 (1417548643094757377), 443 (1417548643095805955), 445 (1417548643101048834), 446 (1417548643107340290), 449 (1417548643109437441)]} 0 68
[junit4:junit4]   2> 16667 T67 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[478 (1417548643047571456), 479 (1417548643053862912), 480 (1417548643073785858), 481 (1417548643087417345), 483 (1417548643089514496), 484 (1417548643092660225), 485 (1417548643097903106), 486 (1417548643106291712), 492 (1417548643109437440), 493 (1417548643110486017)]} 0 62
[junit4:junit4]   2> 16663 T141 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[437 (1417548643044425728), 440 (1417548643045474304), 442 (1417548643052814336), 443 (1417548643054911488), 445 (1417548643055960065), 447 (1417548643061202946), 449 (1417548643084271616), 450 (1417548643088465920), 451 (1417548643094757376), 452 (1417548643100000257)]} 0 60
[junit4:junit4]   2> 16662 T145 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[409 (1417548643039182850), 411 (1417548643041280000), 412 (1417548643047571456), 414 (1417548643051765760), 417 (1417548643055960065), 420 (1417548643072737280), 422 (1417548643087417345), 423 (1417548643091611648), 427 (1417548643094757376), 430 (1417548643102097408)]} 0 64
[junit4:junit4]   2> 16685 T164 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[475 (1417548643045474304), 476 (1417548643053862914), 477 (1417548643062251523), 479 (1417548643065397248), 480 (1417548643072737280), 482 (1417548643075883008), 486 (1417548643081125889), 488 (1417548643107340289), 489 (1417548643109437440), 491 (1417548643126214656)]} 0 82
[junit4:junit4]   2> 16686 T78 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[492 (1417548643047571458), 496 (1417548643052814338), 497 (1417548643053862915), 498 (1417548643080077315), 499 (1417548643083223040), 503 (1417548643097903105), 504 (1417548643102097408), 505 (1417548643107340290), 507 (1417548643126214657), 510 (1417548643129360385)]} 0 80
[junit4:junit4]   2> 16662 T125 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[559 (1417548643057008640), 560 (1417548643074834432), 562 (1417548643082174466), 571 (1417548643085320192), 573 (1417548643088465921), 577 (1417548643090563073), 578 (1417548643092660227), 580 (1417548643094757377), 581 (1417548643097903104), 584 (1417548643100000258)]} 0 54
[junit4:junit4]   2> 16687 T166 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[582 (1417548643066445824), 583 (1417548643073785857), 584 (1417548643080077314), 587 (1417548643092660224), 590 (1417548643095805953), 591 (1417548643107340288), 594 (1417548643125166081), 596 (1417548643127263232), 599 (1417548643129360384), 604 (1417548643131457536)]} 0 64
[junit4:junit4]   2> 16688 T146 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[575 (1417548643049668610), 576 (1417548643058057216), 577 (1417548643077980161), 579 (1417548643088465920), 583 (1417548643089514498), 584 (1417548643107340289), 585 (1417548643109437442), 586 (1417548643126214659), 589 (1417548643129360384), 590 (1417548643131457537)]} 0 80
[junit4:junit4]   2> 16662 T112 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[586 (1417548643063300096), 587 (1417548643085320193), 592 (1417548643086368769), 594 (1417548643091611648), 595 (1417548643093708801), 597 (1417548643096854528), 599 (1417548643097903105), 604 (1417548643100000256), 606 (1417548643101048832), 607 (1417548643105243136)]} 0 41
[junit4:junit4]   2> 16691 T115 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[489 (1417548643047571457), 490 (1417548643058057217), 491 (1417548643080077312), 493 (1417548643106291713), 494 (1417548643109437440), 495 (1417548643126214657), 499 (1417548643129360386), 501 (1417548643131457539), 506 (1417548643133554688), 507 (1417548643134603265)]} 0 86
[junit4:junit4]   2> 16682 T127 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[527 (1417548643049668609), 532 (1417548643079028737), 533 (1417548643087417346), 537 (1417548643092660224), 539 (1417548643096854529), 545 (1417548643098951680), 548 (1417548643108388865), 549 (1417548643110486018), 550 (1417548643124117504), 552 (1417548643126214656)]} 0 74
[junit4:junit4]   2> 16682 T134 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[497 (1417548643042328577), 499 (1417548643058057217), 501 (1417548643061202944), 502 (1417548643073785856), 504 (1417548643082174464), 505 (1417548643083223041), 508 (1417548643106291713), 511 (1417548643108388865), 512 (1417548643111534592), 514 (1417548643126214656)]} 0 141
[junit4:junit4]   2> 16682 T103 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[534 (1417548643047571457), 536 (1417548643052814336), 538 (1417548643054911488), 540 (1417548643058057216), 541 (1417548643062251522), 542 (1417548643073785856), 543 (1417548643076931584), 545 (1417548643082174465), 551 (1417548643110486016), 552 (1417548643112583168)]} 0 77
[junit4:junit4]   2> 16680 T66 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[515 (1417548643039182848), 516 (1417548643041280000), 519 (1417548643055960064), 523 (1417548643064348672), 531 (1417548643066445825), 532 (1417548643079028736), 536 (1417548643092660226), 537 (1417548643098951680), 538 (1417548643105243137), 539 (1417548643110486018)]} 0 83
[junit4:junit4]   2> 16697 T142 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[629 (1417548643129360384), 630 (1417548643131457536), 632 (1417548643131457537), 634 (1417548643132506112), 637 (1417548643133554688), 638 (1417548643134603264), 639 (1417548643136700416), 640 (1417548643138797568), 645 (1417548643139846144), 648 (1417548643141943296)]} 0 13
[junit4:junit4]   2> 16697 T129 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[612 (1417548643110486017), 613 (1417548643123068928), 617 (1417548643126214658), 618 (1417548643129360385), 619 (1417548643131457536), 625 (1417548643132506112), 628 (1417548643133554689), 629 (1417548643136700416), 630 (1417548643138797569), 635 (1417548643140894721)]} 0 32
[junit4:junit4]   2> 16704 T124 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[637 (1417548643125166080), 638 (1417548643129360386), 640 (1417548643131457537), 641 (1417548643133554688), 643 (1417548643136700416), 646 (1417548643138797568), 647 (1417548643140894720), 649 (1417548643142991872), 651 (1417548643146137600), 653 (1417548643148234753)]} 0 25
[junit4:junit4]   2> 16704 T111 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[655 (1417548643127263233), 656 (1417548643130408960), 658 (1417548643132506112), 659 (1417548643133554689), 661 (1417548643134603264), 662 (1417548643137748992), 663 (1417548643139846144), 665 (1417548643141943298), 667 (1417548643144040448), 668 (1417548643147186177)]} 0 22
[junit4:junit4]   2> 16706 T51 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[637 (1417548643131457538), 638 (1417548643132506113), 640 (1417548643134603264), 647 (1417548643135651840), 648 (1417548643137748992), 649 (1417548643138797570), 651 (1417548643140894720), 654 (1417548643145089024), 655 (1417548643149283328), 656 (1417548643150331905)]} 0 20
[junit4:junit4]   2> 16711 T105 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[678 (1417548643139846145), 681 (1417548643141943297), 683 (1417548643142991873), 686 (1417548643146137600), 687 (1417548643147186176), 690 (1417548643148234753), 691 (1417548643149283329), 695 (1417548643150331906), 696 (1417548643152429057), 698 (1417548643155574785)]} 0 17
[junit4:junit4]   2> 16711 T116 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[605 (1417548643135651840), 609 (1417548643138797569), 612 (1417548643141943296), 613 (1417548643145089024), 617 (1417548643147186176), 619 (1417548643148234754), 623 (1417548643149283329), 628 (1417548643151380481), 633 (1417548643153477633), 635 (1417548643155574786)]} 0 22
[junit4:junit4]   2> 16713 T119 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[639 (1417548643130408960), 640 (1417548643132506112), 646 (1417548643133554688), 647 (1417548643134603264), 649 (1417548643136700416), 651 (1417548643138797568), 653 (1417548643145089025), 654 (1417548643148234752), 655 (1417548643150331904), 657 (1417548643153477632)]} 0 28
[junit4:junit4]   2> 16717 T98 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[657 (1417548643136700417), 658 (1417548643138797568), 659 (1417548643139846144), 660 (1417548643141943296), 664 (1417548643142991872), 668 (1417548643145089025), 670 (1417548643148234752), 671 (1417548643150331904), 672 (1417548643152429056), 677 (1417548643154526208)]} 0 26
[junit4:junit4]   2> 16719 T80 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[670 (1417548643141943297), 672 (1417548643148234752), 674 (1417548643149283328), 675 (1417548643150331904), 680 (1417548643153477634), 683 (1417548643155574785), 684 (1417548643156623360), 685 (1417548643157671936), 686 (1417548643160817664), 689 (1417548643162914817)]} 0 23
[junit4:junit4]   2> 16722 T161 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[652 (1417548643137748992), 655 (1417548643139846145), 656 (1417548643141943297), 658 (1417548643145089024), 659 (1417548643147186176), 662 (1417548643150331905), 663 (1417548643153477632), 665 (1417548643155574784), 667 (1417548643160817664), 669 (1417548643161866240)]} 0 29
[junit4:junit4]   2> 16722 T158 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[718 (1417548643152429057), 720 (1417548643154526208), 723 (1417548643155574784), 724 (1417548643157671936), 726 (1417548643158720512), 727 (1417548643160817664), 728 (1417548643161866240), 729 (1417548643162914817), 731 (1417548643163963393), 732 (1417548643165011970)]} 0 16
[junit4:junit4]   2> 16722 T133 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[620 (1417548643137748992), 621 (1417548643139846144), 622 (1417548643146137600), 626 (1417548643148234753), 627 (1417548643153477633), 628 (1417548643157671937), 631 (1417548643160817665), 632 (1417548643162914816), 635 (1417548643165011968), 636 (1417548643167109120)]} 0 31
[junit4:junit4]   2> 16725 T81 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[690 (1417548643148234755), 692 (1417548643151380480), 693 (1417548643153477632), 694 (1417548643155574784), 695 (1417548643160817665), 696 (1417548643162914818), 698 (1417548643165011968), 700 (1417548643167109120), 702 (1417548643168157697), 703 (1417548643170254849)]} 0 23
[junit4:junit4]   2> 16726 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[658 (1417548643147186176), 661 (1417548643152429056), 662 (1417548643156623360), 663 (1417548643158720513), 668 (1417548643160817667), 669 (1417548643163963392), 671 (1417548643166060544), 672 (1417548643168157696), 673 (1417548643169206273), 674 (1417548643172352000)]} 0 25
[junit4:junit4]   2> 16727 T104 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[691 (1417548643146137600), 693 (1417548643148234753), 694 (1417548643149283328), 695 (1417548643151380480), 700 (1417548643156623360), 702 (1417548643157671936), 708 (1417548643159769088), 709 (1417548643162914816), 710 (1417548643169206272), 712 (1417548643171303424)]} 0 26
[junit4:junit4]   2> 16727 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[697 (1417548643147186178), 700 (1417548643150331905), 701 (1417548643153477634), 705 (1417548643157671939), 706 (1417548643160817666), 707 (1417548643162914818), 708 (1417548643165011969), 711 (1417548643167109121), 712 (1417548643171303424), 716 (1417548643173400576)]} 0 26
[junit4:junit4]   2> 16729 T164 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[721 (1417548643159769089), 726 (1417548643160817666), 729 (1417548643162914816), 730 (1417548643163963392), 731 (1417548643167109121), 733 (1417548643168157696), 734 (1417548643169206272), 735 (1417548643170254848), 736 (1417548643172352000), 737 (1417548643174449152)]} 0 16
[junit4:junit4]   2> 16729 T144 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[675 (1417548643145089024), 678 (1417548643147186177), 681 (1417548643148234754), 682 (1417548643154526209), 684 (1417548643157671938), 685 (1417548643169206272), 688 (1417548643170254848), 691 (1417548643172352001), 693 (1417548643174449152), 694 (1417548643175497728)]} 0 31
[junit4:junit4]   2> 16730 T148 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[705 (1417548643152429056), 707 (1417548643154526208), 709 (1417548643156623361), 710 (1417548643159769088), 711 (1417548643160817667), 713 (1417548643163963393), 714 (1417548643169206273), 715 (1417548643171303424), 717 (1417548643173400576), 718 (1417548643175497728)]} 0 24
[junit4:junit4]   2> 16731 T52 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[699 (1417548643152429058), 704 (1417548643155574784), 705 (1417548643160817665), 706 (1417548643162914816), 707 (1417548643166060544), 708 (1417548643167109120), 709 (1417548643168157696), 710 (1417548643170254848), 711 (1417548643172352000), 712 (1417548643174449152)]} 0 25
[junit4:junit4]   2> 16739 T65 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[670 (1417548643148234752), 672 (1417548643150331904), 674 (1417548643155574785), 676 (1417548643162914817), 683 (1417548643163963393), 685 (1417548643165011968), 686 (1417548643172352000), 688 (1417548643174449152), 689 (1417548643176546304), 690 (1417548643180740608)]} 0 38
[junit4:junit4]   2> 16741 T128 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[714 (1417548643156623360), 720 (1417548643157671936), 721 (1417548643160817664), 722 (1417548643161866240), 725 (1417548643169206272), 726 (1417548643171303424), 727 (1417548643173400576), 728 (1417548643175497729), 733 (1417548643178643456), 734 (1417548643185983488)]} 0 31
[junit4:junit4]   2> 16743 T160 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[717 (1417548643160817665), 718 (1417548643163963392), 721 (1417548643166060544), 722 (1417548643168157696), 724 (1417548643170254848), 726 (1417548643175497728), 728 (1417548643178643456), 732 (1417548643182837761), 735 (1417548643188080640), 737 (1417548643190177792)]} 0 29
[junit4:junit4]   2> 16749 T145 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[736 (1417548643173400577), 740 (1417548643174449153), 742 (1417548643175497728), 743 (1417548643177594880), 744 (1417548643181789184), 745 (1417548643187032064), 748 (1417548643190177792), 751 (1417548643192274944), 754 (1417548643193323521), 755 (1417548643195420672)]} 0 23
[junit4:junit4]   2> 16754 T53 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[793 (1417548643189129216), 794 (1417548643190177793), 795 (1417548643191226369), 796 (1417548643193323520), 798 (1417548643194372096), 799 (1417548643195420673), 801 (1417548643197517824), 805 (1417548643198566401), 808 (1417548643199614976), 809 (1417548643200663553)]} 0 13
[junit4:junit4]   2> 16755 T121 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[742 (1417548643175497729), 743 (1417548643176546305), 747 (1417548643181789184), 748 (1417548643182837760), 749 (1417548643188080641), 750 (1417548643190177793), 751 (1417548643192274945), 752 (1417548643194372096), 753 (1417548643196469248), 754 (1417548643198566401)]} 0 27
[junit4:junit4]   2> 16757 T149 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[753 (1417548643185983489), 755 (1417548643188080641), 756 (1417548643190177793), 757 (1417548643192274944), 758 (1417548643193323520), 760 (1417548643195420672), 762 (1417548643197517824), 763 (1417548643199614977), 765 (1417548643201712128), 767 (1417548643203809280)]} 0 22
[junit4:junit4]   2> 16758 T143 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[733 (1417548643185983488), 736 (1417548643188080640), 738 (1417548643190177793), 739 (1417548643192274944), 740 (1417548643194372097), 745 (1417548643196469249), 750 (1417548643198566403), 754 (1417548643200663553), 764 (1417548643202760705), 765 (1417548643204857857)]} 0 20
[junit4:junit4]   2> 16760 T78 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[738 (1417548643181789184), 741 (1417548643185983488), 742 (1417548643187032064), 744 (1417548643188080642), 745 (1417548643191226368), 747 (1417548643194372096), 748 (1417548643196469248), 750 (1417548643199614976), 751 (1417548643201712129), 752 (1417548643205906433)]} 0 26
[junit4:junit4]   2> 16762 T113 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[766 (1417548643190177792), 771 (1417548643192274945), 772 (1417548643194372096), 773 (1417548643196469248), 778 (1417548643198566402), 779 (1417548643200663552), 781 (1417548643201712129), 782 (1417548643204857856), 784 (1417548643206955008), 785 (1417548643209052160)]} 0 20
[junit4:junit4]   2> 16765 T159 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[773 (1417548643188080642), 774 (1417548643190177794), 781 (1417548643192274944), 782 (1417548643193323520), 783 (1417548643195420672), 784 (1417548643197517824), 786 (1417548643200663552), 787 (1417548643202760704), 788 (1417548643204857856), 789 (1417548643212197888)]} 0 25
[junit4:junit4]   2> 16768 T102 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[786 (1417548643195420673), 788 (1417548643198566400), 790 (1417548643199614977), 791 (1417548643201712128), 794 (1417548643203809280), 795 (1417548643206955009), 797 (1417548643209052161), 798 (1417548643211149312), 800 (1417548643213246464), 802 (1417548643215343616)]} 0 21
[junit4:junit4]   2> 16768 T166 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[794 (1417548643191226369), 798 (1417548643193323521), 799 (1417548643196469249), 801 (1417548643199614978), 802 (1417548643201712130), 804 (1417548643204857856), 806 (1417548643206955008), 807 (14175486432101

[...truncated too long message...]

dThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene41: {timestamp=PostingsFormat(name=Direct), range_facet_l=PostingsFormat(name=Direct), id=Pulsing41(freqCutoff=2 minBlockSize=50 maxBlockSize=174), range_facet_sl=PostingsFormat(name=Lucene41WithOrds), range_facet_si=MockFixedIntBlock(blockSize=1544), multiDefault=MockFixedIntBlock(blockSize=1544), intDefault=PostingsFormat(name=Direct)}, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=es_GT, timezone=Pacific/Truk
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=5,free=77576888,total=312475648
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestFoldingMultitermQuery, IndexSchemaRuntimeFieldTest, DirectSolrSpellCheckerTest, RAMDirectoryFactoryTest, StatsComponentTest, TestSurroundQueryParser, RequiredFieldsTest, DocumentAnalysisRequestHandlerTest, BadIndexSchemaTest, TestSolrDeletionPolicy1, TestWriterPerf, TestCSVResponseWriter, DOMUtilTest, StatelessScriptUpdateProcessorFactoryTest, ZkCLITest, DebugComponentTest, LeaderElectionTest, TestReversedWildcardFilterFactory, FieldMutatingUpdateProcessorTest, TestTrie, TestRemoteStreaming]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestRemoteStreaming -Dtests.seed=8E53F6B1A0D0F2CF -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_GT -Dtests.timezone=Pacific/Truk -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] ERROR   0.00s J1 | TestRemoteStreaming (suite) <<<
[junit4:junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming: 
[junit4:junit4]    >    1) Thread[id=297, name=qtp1321261771-297 Acceptor2 SelectChannelConnector@0.0.0.0:58147, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=298, name=qtp1321261771-298 Acceptor3 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=296, name=qtp1321261771-296 Acceptor1 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=295, name=qtp1321261771-295 Acceptor0 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([8E53F6B1A0D0F2CF]:0)
[junit4:junit4]    > Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4]    >    1) Thread[id=297, name=qtp1321261771-297 Acceptor2 SelectChannelConnector@0.0.0.0:58147, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=298, name=qtp1321261771-298 Acceptor3 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=296, name=qtp1321261771-296 Acceptor1 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=295, name=qtp1321261771-295 Acceptor0 SelectChannelConnector@0.0.0.0:58147, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([8E53F6B1A0D0F2CF]:0)
[junit4:junit4] Completed on J1 in 26.42s, 4 tests, 2 errors <<< FAILURES!

[...truncated 122 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:335: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/module-build.xml:61: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:1163: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:827: There were test failures: 233 suites, 965 tests, 4 suite-level errors, 777 ignored

Total time: 18 minutes 35 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 616 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/616/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.TestDistributedGrouping

Error Message:
36 threads leaked from SUITE scope at org.apache.solr.TestDistributedGrouping:     1) Thread[id=167, name=qtp1514649449-167 Acceptor2 SelectChannelConnector@0.0.0.0:19908, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=88, name=qtp571201483-88 Acceptor3 SelectChannelConnector@0.0.0.0:19892, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=73, name=qtp1799456698-73 Acceptor2 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=39, name=qtp1340662491-39 Acceptor3 SelectChannelConnector@0.0.0.0:16514, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=155, name=qtp109963104-155 Acceptor3 SelectChannelConnector@0.0.0.0:19906, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=100, name=qtp1318450927-100 Acceptor2 SelectChannelConnector@0.0.0.0:19893, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=37, name=qtp1340662491-37 Acceptor1 SelectChannelConnector@0.0.0.0:16514, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=50, name=qtp1832089855-50 Acceptor0 SelectChannelConnector@0.0.0.0:19879, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=140, name=qtp29394673-140 Acceptor2 SelectChannelConnector@0.0.0.0:19905, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=124, name=qtp1017859891-124 Acceptor0 SelectChannelConnector@0.0.0.0:19904, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=87, name=qtp571201483-87 Acceptor2 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=154, name=qtp109963104-154 Acceptor2 SelectChannelConnector@0.0.0.0:19906, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=168, name=qtp1514649449-168 Acceptor3 SelectChannelConnector@0.0.0.0:19908, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=139, name=qtp29394673-139 Acceptor1 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=99, name=qtp1318450927-99 Acceptor1 SelectChannelConnector@0.0.0.0:19893, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=51, name=qtp1832089855-51 Acceptor1 SelectChannelConnector@0.0.0.0:19879, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=38, name=qtp1340662491-38 Acceptor2 SelectChannelConnector@0.0.0.0:16514, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=165, name=qtp1514649449-165 Acceptor0 SelectChannelConnector@0.0.0.0:19908, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=152, name=qtp109963104-152 Acceptor0 SelectChannelConnector@0.0.0.0:19906, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=126, name=qtp1017859891-126 Acceptor2 SelectChannelConnector@0.0.0.0:19904, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   21) Thread[id=153, name=qtp109963104-153 Acceptor1 SelectChannelConnector@0.0.0.0:19906, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   22) Thread[id=98, name=qtp1318450927-98 Acceptor0 SelectChannelConnector@0.0.0.0:19893, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   23) Thread[id=36, name=qtp1340662491-36 Acceptor0 SelectChannelConnector@0.0.0.0:16514, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   24) Thread[id=53, name=qtp1832089855-53 Acceptor3 SelectChannelConnector@0.0.0.0:19879, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   25) Thread[id=101, name=qtp1318450927-101 Acceptor3 SelectChannelConnector@0.0.0.0:19893, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   26) Thread[id=127, name=qtp1017859891-127 Acceptor3 SelectChannelConnector@0.0.0.0:19904, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   27) Thread[id=71, name=qtp1799456698-71 Acceptor0 SelectChannelConnector@0.0.0.0:19889, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   28) Thread[id=166, name=qtp1514649449-166 Acceptor1 SelectChannelConnector@0.0.0.0:19908, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   29) Thread[id=72, name=qtp1799456698-72 Acceptor1 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   30) Thread[id=141, name=qtp29394673-141 Acceptor3 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   31) Thread[id=86, name=qtp571201483-86 Acceptor1 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   32) Thread[id=85, name=qtp571201483-85 Acceptor0 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   33) Thread[id=52, name=qtp1832089855-52 Acceptor2 SelectChannelConnector@0.0.0.0:19879, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   34) Thread[id=138, name=qtp29394673-138 Acceptor0 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   35) Thread[id=125, name=qtp1017859891-125 Acceptor1 SelectChannelConnector@0.0.0.0:19904, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   36) Thread[id=74, name=qtp1799456698-74 Acceptor3 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 36 threads leaked from SUITE scope at org.apache.solr.TestDistributedGrouping: 
   1) Thread[id=167, name=qtp1514649449-167 Acceptor2 SelectChannelConnector@0.0.0.0:19908, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=88, name=qtp571201483-88 Acceptor3 SelectChannelConnector@0.0.0.0:19892, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=73, name=qtp1799456698-73 Acceptor2 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=39, name=qtp1340662491-39 Acceptor3 SelectChannelConnector@0.0.0.0:16514, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=155, name=qtp109963104-155 Acceptor3 SelectChannelConnector@0.0.0.0:19906, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=100, name=qtp1318450927-100 Acceptor2 SelectChannelConnector@0.0.0.0:19893, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=37, name=qtp1340662491-37 Acceptor1 SelectChannelConnector@0.0.0.0:16514, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=50, name=qtp1832089855-50 Acceptor0 SelectChannelConnector@0.0.0.0:19879, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=140, name=qtp29394673-140 Acceptor2 SelectChannelConnector@0.0.0.0:19905, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=124, name=qtp1017859891-124 Acceptor0 SelectChannelConnector@0.0.0.0:19904, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=87, name=qtp571201483-87 Acceptor2 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=154, name=qtp109963104-154 Acceptor2 SelectChannelConnector@0.0.0.0:19906, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=168, name=qtp1514649449-168 Acceptor3 SelectChannelConnector@0.0.0.0:19908, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=139, name=qtp29394673-139 Acceptor1 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=99, name=qtp1318450927-99 Acceptor1 SelectChannelConnector@0.0.0.0:19893, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=51, name=qtp1832089855-51 Acceptor1 SelectChannelConnector@0.0.0.0:19879, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=38, name=qtp1340662491-38 Acceptor2 SelectChannelConnector@0.0.0.0:16514, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=165, name=qtp1514649449-165 Acceptor0 SelectChannelConnector@0.0.0.0:19908, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=152, name=qtp109963104-152 Acceptor0 SelectChannelConnector@0.0.0.0:19906, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=126, name=qtp1017859891-126 Acceptor2 SelectChannelConnector@0.0.0.0:19904, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  21) Thread[id=153, name=qtp109963104-153 Acceptor1 SelectChannelConnector@0.0.0.0:19906, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  22) Thread[id=98, name=qtp1318450927-98 Acceptor0 SelectChannelConnector@0.0.0.0:19893, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  23) Thread[id=36, name=qtp1340662491-36 Acceptor0 SelectChannelConnector@0.0.0.0:16514, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  24) Thread[id=53, name=qtp1832089855-53 Acceptor3 SelectChannelConnector@0.0.0.0:19879, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  25) Thread[id=101, name=qtp1318450927-101 Acceptor3 SelectChannelConnector@0.0.0.0:19893, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  26) Thread[id=127, name=qtp1017859891-127 Acceptor3 SelectChannelConnector@0.0.0.0:19904, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  27) Thread[id=71, name=qtp1799456698-71 Acceptor0 SelectChannelConnector@0.0.0.0:19889, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  28) Thread[id=166, name=qtp1514649449-166 Acceptor1 SelectChannelConnector@0.0.0.0:19908, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  29) Thread[id=72, name=qtp1799456698-72 Acceptor1 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  30) Thread[id=141, name=qtp29394673-141 Acceptor3 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  31) Thread[id=86, name=qtp571201483-86 Acceptor1 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  32) Thread[id=85, name=qtp571201483-85 Acceptor0 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  33) Thread[id=52, name=qtp1832089855-52 Acceptor2 SelectChannelConnector@0.0.0.0:19879, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  34) Thread[id=138, name=qtp29394673-138 Acceptor0 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  35) Thread[id=125, name=qtp1017859891-125 Acceptor1 SelectChannelConnector@0.0.0.0:19904, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  36) Thread[id=74, name=qtp1799456698-74 Acceptor3 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([9A74CB360EB6D5D0]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.TestDistributedGrouping

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=167, name=qtp1514649449-167 Acceptor2 SelectChannelConnector@0.0.0.0:19908, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=88, name=qtp571201483-88 Acceptor3 SelectChannelConnector@0.0.0.0:19892, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=73, name=qtp1799456698-73 Acceptor2 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=39, name=qtp1340662491-39 Acceptor3 SelectChannelConnector@0.0.0.0:16514, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=155, name=qtp109963104-155 Acceptor3 SelectChannelConnector@0.0.0.0:19906, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=100, name=qtp1318450927-100 Acceptor2 SelectChannelConnector@0.0.0.0:19893, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=37, name=qtp1340662491-37 Acceptor1 SelectChannelConnector@0.0.0.0:16514, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=50, name=qtp1832089855-50 Acceptor0 SelectChannelConnector@0.0.0.0:19879, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=140, name=qtp29394673-140 Acceptor2 SelectChannelConnector@0.0.0.0:19905, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=124, name=qtp1017859891-124 Acceptor0 SelectChannelConnector@0.0.0.0:19904, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=87, name=qtp571201483-87 Acceptor2 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=154, name=qtp109963104-154 Acceptor2 SelectChannelConnector@0.0.0.0:19906, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=139, name=qtp29394673-139 Acceptor1 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=168, name=qtp1514649449-168 Acceptor3 SelectChannelConnector@0.0.0.0:19908, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=38, name=qtp1340662491-38 Acceptor2 SelectChannelConnector@0.0.0.0:16514, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=51, name=qtp1832089855-51 Acceptor1 SelectChannelConnector@0.0.0.0:19879, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=99, name=qtp1318450927-99 Acceptor1 SelectChannelConnector@0.0.0.0:19893, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=165, name=qtp1514649449-165 Acceptor0 SelectChannelConnector@0.0.0.0:19908, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=152, name=qtp109963104-152 Acceptor0 SelectChannelConnector@0.0.0.0:19906, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=126, name=qtp1017859891-126 Acceptor2 SelectChannelConnector@0.0.0.0:19904, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   21) Thread[id=98, name=qtp1318450927-98 Acceptor0 SelectChannelConnector@0.0.0.0:19893, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   22) Thread[id=153, name=qtp109963104-153 Acceptor1 SelectChannelConnector@0.0.0.0:19906, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   23) Thread[id=36, name=qtp1340662491-36 Acceptor0 SelectChannelConnector@0.0.0.0:16514, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   24) Thread[id=53, name=qtp1832089855-53 Acceptor3 SelectChannelConnector@0.0.0.0:19879, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   25) Thread[id=101, name=qtp1318450927-101 Acceptor3 SelectChannelConnector@0.0.0.0:19893, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   26) Thread[id=127, name=qtp1017859891-127 Acceptor3 SelectChannelConnector@0.0.0.0:19904, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   27) Thread[id=71, name=qtp1799456698-71 Acceptor0 SelectChannelConnector@0.0.0.0:19889, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   28) Thread[id=166, name=qtp1514649449-166 Acceptor1 SelectChannelConnector@0.0.0.0:19908, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   29) Thread[id=72, name=qtp1799456698-72 Acceptor1 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   30) Thread[id=141, name=qtp29394673-141 Acceptor3 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   31) Thread[id=86, name=qtp571201483-86 Acceptor1 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   32) Thread[id=85, name=qtp571201483-85 Acceptor0 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   33) Thread[id=52, name=qtp1832089855-52 Acceptor2 SelectChannelConnector@0.0.0.0:19879, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   34) Thread[id=138, name=qtp29394673-138 Acceptor0 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   35) Thread[id=125, name=qtp1017859891-125 Acceptor1 SelectChannelConnector@0.0.0.0:19904, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   36) Thread[id=74, name=qtp1799456698-74 Acceptor3 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=167, name=qtp1514649449-167 Acceptor2 SelectChannelConnector@0.0.0.0:19908, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=88, name=qtp571201483-88 Acceptor3 SelectChannelConnector@0.0.0.0:19892, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=73, name=qtp1799456698-73 Acceptor2 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=39, name=qtp1340662491-39 Acceptor3 SelectChannelConnector@0.0.0.0:16514, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=155, name=qtp109963104-155 Acceptor3 SelectChannelConnector@0.0.0.0:19906, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=100, name=qtp1318450927-100 Acceptor2 SelectChannelConnector@0.0.0.0:19893, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=37, name=qtp1340662491-37 Acceptor1 SelectChannelConnector@0.0.0.0:16514, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=50, name=qtp1832089855-50 Acceptor0 SelectChannelConnector@0.0.0.0:19879, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=140, name=qtp29394673-140 Acceptor2 SelectChannelConnector@0.0.0.0:19905, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=124, name=qtp1017859891-124 Acceptor0 SelectChannelConnector@0.0.0.0:19904, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=87, name=qtp571201483-87 Acceptor2 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=154, name=qtp109963104-154 Acceptor2 SelectChannelConnector@0.0.0.0:19906, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=139, name=qtp29394673-139 Acceptor1 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=168, name=qtp1514649449-168 Acceptor3 SelectChannelConnector@0.0.0.0:19908, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=38, name=qtp1340662491-38 Acceptor2 SelectChannelConnector@0.0.0.0:16514, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=51, name=qtp1832089855-51 Acceptor1 SelectChannelConnector@0.0.0.0:19879, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=99, name=qtp1318450927-99 Acceptor1 SelectChannelConnector@0.0.0.0:19893, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=165, name=qtp1514649449-165 Acceptor0 SelectChannelConnector@0.0.0.0:19908, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=152, name=qtp109963104-152 Acceptor0 SelectChannelConnector@0.0.0.0:19906, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=126, name=qtp1017859891-126 Acceptor2 SelectChannelConnector@0.0.0.0:19904, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  21) Thread[id=98, name=qtp1318450927-98 Acceptor0 SelectChannelConnector@0.0.0.0:19893, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  22) Thread[id=153, name=qtp109963104-153 Acceptor1 SelectChannelConnector@0.0.0.0:19906, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  23) Thread[id=36, name=qtp1340662491-36 Acceptor0 SelectChannelConnector@0.0.0.0:16514, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  24) Thread[id=53, name=qtp1832089855-53 Acceptor3 SelectChannelConnector@0.0.0.0:19879, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  25) Thread[id=101, name=qtp1318450927-101 Acceptor3 SelectChannelConnector@0.0.0.0:19893, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  26) Thread[id=127, name=qtp1017859891-127 Acceptor3 SelectChannelConnector@0.0.0.0:19904, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  27) Thread[id=71, name=qtp1799456698-71 Acceptor0 SelectChannelConnector@0.0.0.0:19889, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  28) Thread[id=166, name=qtp1514649449-166 Acceptor1 SelectChannelConnector@0.0.0.0:19908, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  29) Thread[id=72, name=qtp1799456698-72 Acceptor1 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  30) Thread[id=141, name=qtp29394673-141 Acceptor3 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  31) Thread[id=86, name=qtp571201483-86 Acceptor1 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  32) Thread[id=85, name=qtp571201483-85 Acceptor0 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  33) Thread[id=52, name=qtp1832089855-52 Acceptor2 SelectChannelConnector@0.0.0.0:19879, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  34) Thread[id=138, name=qtp29394673-138 Acceptor0 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  35) Thread[id=125, name=qtp1017859891-125 Acceptor1 SelectChannelConnector@0.0.0.0:19904, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  36) Thread[id=74, name=qtp1799456698-74 Acceptor3 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([9A74CB360EB6D5D0]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.update.SolrCmdDistributorTest

Error Message:
20 threads leaked from SUITE scope at org.apache.solr.update.SolrCmdDistributorTest:     1) Thread[id=65, name=qtp1096973217-65 Acceptor2 SelectChannelConnector@0.0.0.0:12325, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=49, name=qtp117369574-49 Acceptor0 SelectChannelConnector@0.0.0.0:33200, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=22, name=qtp1496026375-22 Acceptor1 SelectChannelConnector@0.0.0.0:58061, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=52, name=qtp117369574-52 Acceptor3 SelectChannelConnector@0.0.0.0:33200, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=77, name=qtp479278366-77 Acceptor0 SelectChannelConnector@0.0.0.0:35144, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=23, name=qtp1496026375-23 Acceptor2 SelectChannelConnector@0.0.0.0:58061, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=66, name=qtp1096973217-66 Acceptor3 SelectChannelConnector@0.0.0.0:12325, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=35, name=qtp316387877-35 Acceptor0 SelectChannelConnector@0.0.0.0:25933, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=21, name=qtp1496026375-21 Acceptor0 SelectChannelConnector@0.0.0.0:58061, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=64, name=qtp1096973217-64 Acceptor1 SelectChannelConnector@0.0.0.0:12325, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=36, name=qtp316387877-36 Acceptor1 SelectChannelConnector@0.0.0.0:25933, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=79, name=qtp479278366-79 Acceptor2 SelectChannelConnector@0.0.0.0:35144, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=63, name=qtp1096973217-63 Acceptor0 SelectChannelConnector@0.0.0.0:12325, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=51, name=qtp117369574-51 Acceptor2 SelectChannelConnector@0.0.0.0:33200, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=37, name=qtp316387877-37 Acceptor2 SelectChannelConnector@0.0.0.0:25933, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=38, name=qtp316387877-38 Acceptor3 SelectChannelConnector@0.0.0.0:25933, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=50, name=qtp117369574-50 Acceptor1 SelectChannelConnector@0.0.0.0:33200, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=78, name=qtp479278366-78 Acceptor1 SelectChannelConnector@0.0.0.0:35144, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=24, name=qtp1496026375-24 Acceptor3 SelectChannelConnector@0.0.0.0:58061, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=80, name=qtp479278366-80 Acceptor3 SelectChannelConnector@0.0.0.0:35144, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 20 threads leaked from SUITE scope at org.apache.solr.update.SolrCmdDistributorTest: 
   1) Thread[id=65, name=qtp1096973217-65 Acceptor2 SelectChannelConnector@0.0.0.0:12325, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=49, name=qtp117369574-49 Acceptor0 SelectChannelConnector@0.0.0.0:33200, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=22, name=qtp1496026375-22 Acceptor1 SelectChannelConnector@0.0.0.0:58061, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=52, name=qtp117369574-52 Acceptor3 SelectChannelConnector@0.0.0.0:33200, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=77, name=qtp479278366-77 Acceptor0 SelectChannelConnector@0.0.0.0:35144, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=23, name=qtp1496026375-23 Acceptor2 SelectChannelConnector@0.0.0.0:58061, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=66, name=qtp1096973217-66 Acceptor3 SelectChannelConnector@0.0.0.0:12325, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=35, name=qtp316387877-35 Acceptor0 SelectChannelConnector@0.0.0.0:25933, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=21, name=qtp1496026375-21 Acceptor0 SelectChannelConnector@0.0.0.0:58061, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=64, name=qtp1096973217-64 Acceptor1 SelectChannelConnector@0.0.0.0:12325, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=36, name=qtp316387877-36 Acceptor1 SelectChannelConnector@0.0.0.0:25933, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=79, name=qtp479278366-79 Acceptor2 SelectChannelConnector@0.0.0.0:35144, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=63, name=qtp1096973217-63 Acceptor0 SelectChannelConnector@0.0.0.0:12325, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=51, name=qtp117369574-51 Acceptor2 SelectChannelConnector@0.0.0.0:33200, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=37, name=qtp316387877-37 Acceptor2 SelectChannelConnector@0.0.0.0:25933, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=38, name=qtp316387877-38 Acceptor3 SelectChannelConnector@0.0.0.0:25933, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=50, name=qtp117369574-50 Acceptor1 SelectChannelConnector@0.0.0.0:33200, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=78, name=qtp479278366-78 Acceptor1 SelectChannelConnector@0.0.0.0:35144, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=24, name=qtp1496026375-24 Acceptor3 SelectChannelConnector@0.0.0.0:58061, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=80, name=qtp479278366-80 Acceptor3 SelectChannelConnector@0.0.0.0:35144, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([9A74CB360EB6D5D0]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.update.SolrCmdDistributorTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=65, name=qtp1096973217-65 Acceptor2 SelectChannelConnector@0.0.0.0:12325, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=49, name=qtp117369574-49 Acceptor0 SelectChannelConnector@0.0.0.0:33200, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=22, name=qtp1496026375-22 Acceptor1 SelectChannelConnector@0.0.0.0:58061, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=52, name=qtp117369574-52 Acceptor3 SelectChannelConnector@0.0.0.0:33200, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=23, name=qtp1496026375-23 Acceptor2 SelectChannelConnector@0.0.0.0:58061, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=77, name=qtp479278366-77 Acceptor0 SelectChannelConnector@0.0.0.0:35144, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=66, name=qtp1096973217-66 Acceptor3 SelectChannelConnector@0.0.0.0:12325, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=21, name=qtp1496026375-21 Acceptor0 SelectChannelConnector@0.0.0.0:58061, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=35, name=qtp316387877-35 Acceptor0 SelectChannelConnector@0.0.0.0:25933, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=64, name=qtp1096973217-64 Acceptor1 SelectChannelConnector@0.0.0.0:12325, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=36, name=qtp316387877-36 Acceptor1 SelectChannelConnector@0.0.0.0:25933, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=63, name=qtp1096973217-63 Acceptor0 SelectChannelConnector@0.0.0.0:12325, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=79, name=qtp479278366-79 Acceptor2 SelectChannelConnector@0.0.0.0:35144, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=51, name=qtp117369574-51 Acceptor2 SelectChannelConnector@0.0.0.0:33200, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=37, name=qtp316387877-37 Acceptor2 SelectChannelConnector@0.0.0.0:25933, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=38, name=qtp316387877-38 Acceptor3 SelectChannelConnector@0.0.0.0:25933, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=50, name=qtp117369574-50 Acceptor1 SelectChannelConnector@0.0.0.0:33200, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=78, name=qtp479278366-78 Acceptor1 SelectChannelConnector@0.0.0.0:35144, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=24, name=qtp1496026375-24 Acceptor3 SelectChannelConnector@0.0.0.0:58061, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=80, name=qtp479278366-80 Acceptor3 SelectChannelConnector@0.0.0.0:35144, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=65, name=qtp1096973217-65 Acceptor2 SelectChannelConnector@0.0.0.0:12325, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=49, name=qtp117369574-49 Acceptor0 SelectChannelConnector@0.0.0.0:33200, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=22, name=qtp1496026375-22 Acceptor1 SelectChannelConnector@0.0.0.0:58061, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=52, name=qtp117369574-52 Acceptor3 SelectChannelConnector@0.0.0.0:33200, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=23, name=qtp1496026375-23 Acceptor2 SelectChannelConnector@0.0.0.0:58061, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=77, name=qtp479278366-77 Acceptor0 SelectChannelConnector@0.0.0.0:35144, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=66, name=qtp1096973217-66 Acceptor3 SelectChannelConnector@0.0.0.0:12325, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=21, name=qtp1496026375-21 Acceptor0 SelectChannelConnector@0.0.0.0:58061, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=35, name=qtp316387877-35 Acceptor0 SelectChannelConnector@0.0.0.0:25933, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=64, name=qtp1096973217-64 Acceptor1 SelectChannelConnector@0.0.0.0:12325, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=36, name=qtp316387877-36 Acceptor1 SelectChannelConnector@0.0.0.0:25933, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=63, name=qtp1096973217-63 Acceptor0 SelectChannelConnector@0.0.0.0:12325, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=79, name=qtp479278366-79 Acceptor2 SelectChannelConnector@0.0.0.0:35144, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=51, name=qtp117369574-51 Acceptor2 SelectChannelConnector@0.0.0.0:33200, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=37, name=qtp316387877-37 Acceptor2 SelectChannelConnector@0.0.0.0:25933, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=38, name=qtp316387877-38 Acceptor3 SelectChannelConnector@0.0.0.0:25933, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=50, name=qtp117369574-50 Acceptor1 SelectChannelConnector@0.0.0.0:33200, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=78, name=qtp479278366-78 Acceptor1 SelectChannelConnector@0.0.0.0:35144, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=24, name=qtp1496026375-24 Acceptor3 SelectChannelConnector@0.0.0.0:58061, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=80, name=qtp479278366-80 Acceptor3 SelectChannelConnector@0.0.0.0:35144, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([9A74CB360EB6D5D0]:0)




Build Log:
[...truncated 8300 lines...]
[junit4:junit4] Suite: org.apache.solr.update.SolrCmdDistributorTest
[junit4:junit4]   2> 7 T14 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> 457 T14 oejs.Server.doStart jetty-8.1.7.v20120910
[junit4:junit4]   2> 754 T14 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:58061
[junit4:junit4]   2> 842 T14 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 843 T14 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 843 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 905 T14 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 906 T14 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 907 T14 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 908 T14 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 908 T14 oasc.CoreContainer.<init> New CoreContainer 1143585732
[junit4:junit4]   2> 910 T14 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 910 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 959 T14 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 1028 T14 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 1029 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 1031 T14 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 1032 T14 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 1102 T14 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 1162 T14 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 1163 T14 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 1214 T14 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1829 T14 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1843 T14 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1847 T14 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1867 T14 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1874 T14 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1879 T14 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1882 T14 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1885 T14 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1352461659291/control/data/
[junit4:junit4]   2> 1886 T14 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 1888 T14 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1352461659291/control/data/index/
[junit4:junit4]   2> 1889 T14 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1352461659291/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1890 T14 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1352461659291/control/data/index forceNew:false
[junit4:junit4]   2> 1900 T14 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1ea046df lockFactory=org.apache.lucene.store.NativeFSLockFactory@576288db),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1901 T14 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 1938 T14 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1963 T14 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1984 T14 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1990 T14 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2006 T14 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2013 T14 oass.SolrIndexSearcher.<init> Opening Searcher@89e364f main
[junit4:junit4]   2> 2014 T14 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 2081 T14 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2081 T14 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2093 T14 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 2094 T14 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 2094 T14 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 2095 T14 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 2095 T14 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 2096 T14 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 2096 T14 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 2097 T14 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 2098 T14 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 2148 T14 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 2704 T29 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@89e364f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2707 T14 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 2708 T14 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 2709 T14 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 2766 T14 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 2789 T14 oejs.Server.doStart jetty-8.1.7.v20120910
[junit4:junit4]   2> 2792 T14 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:25933
[junit4:junit4]   2> 2793 T14 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2794 T14 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 2794 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2830 T14 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 2831 T14 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2831 T14 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 2832 T14 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 2832 T14 oasc.CoreContainer.<init> New CoreContainer 746137641
[junit4:junit4]   2> 2833 T14 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2834 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2863 T14 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 2883 T14 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 2884 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 2886 T14 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 2887 T14 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 2942 T14 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 2993 T14 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 2994 T14 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 3041 T14 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 3602 T14 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 3616 T14 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 3619 T14 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 3640 T14 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 3645 T14 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 3650 T14 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 3652 T14 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 3655 T14 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1352461659291/shard0/data/
[junit4:junit4]   2> 3655 T14 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 3656 T14 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1352461659291/shard0/data/index/
[junit4:junit4]   2> 3657 T14 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1352461659291/shard0/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 3673 T14 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1352461659291/shard0/data/index forceNew:false
[junit4:junit4]   2> 3677 T14 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@1d5c9aa6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@320c4843)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 3677 T14 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 3679 T14 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 3680 T14 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 3680 T14 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 3681 T14 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 3691 T14 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 3697 T14 oass.SolrIndexSearcher.<init> Opening Searcher@6181ab1 main
[junit4:junit4]   2> 3697 T14 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3699 T14 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 3699 T14 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 3700 T14 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 3701 T14 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 3701 T14 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 3702 T14 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 3702 T14 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 3703 T14 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 3703 T14 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 3704 T14 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 3704 T14 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 3705 T14 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 3719 T43 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6181ab1 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 3719 T14 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 3720 T14 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 3721 T14 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3721 T14 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3732 T14 oejs.Server.doStart jetty-8.1.7.v20120910
[junit4:junit4]   2> 3735 T14 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:33200
[junit4:junit4]   2> 3736 T14 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3737 T14 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 3737 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 3766 T14 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 3767 T14 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3768 T14 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 3768 T14 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 3769 T14 oasc.CoreContainer.<init> New CoreContainer 167283382
[junit4:junit4]   2> 3769 T14 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 3770 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 3796 T14 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 3813 T14 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 3814 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 3816 T14 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 3817 T14 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 3855 T14 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 3898 T14 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 3900 T14 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 3939 T14 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 4492 T14 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 4511 T14 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 4514 T14 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 4534 T14 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4539 T14 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4544 T14 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4547 T14 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4550 T14 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1352461659291/shard1/data/
[junit4:junit4]   2> 4551 T14 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 4553 T14 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1352461659291/shard1/data/index/
[junit4:junit4]   2> 4554 T14 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1352461659291/shard1/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 4556 T14 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1352461659291/shard1/data/index forceNew:false
[junit4:junit4]   2> 4560 T14 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@79e70822 lockFactory=org.apache.lucene.store.NativeFSLockFactory@16cb436a),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 4561 T14 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 4563 T14 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 4564 T14 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 4565 T14 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 4566 T14 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 4582 T14 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 4593 T14 oass.SolrIndexSearcher.<init> Opening Searcher@43d06911 main
[junit4:junit4]   2> 4594 T14 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 4596 T14 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 4597 T14 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 4598 T14 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 4599 T14 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4599 T14 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 4600 T14 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4601 T14 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4602 T14 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4602 T14 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4603 T14 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4604 T14 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4605 T14 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 4628 T57 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@43d06911 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 4628 T14 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 4629 T14 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 4630 T14 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 4631 T14 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 4646 T14 oejs.Server.doStart jetty-8.1.7.v20120910
[junit4:junit4]   2> 4650 T14 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:12325
[junit4:junit4]   2> 4652 T14 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4653 T14 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 4653 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4698 T14 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 4699 T14 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4700 T14 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 4701 T14 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 4701 T14 oasc.CoreContainer.<init> New CoreContainer 1620817728
[junit4:junit4]   2> 4703 T14 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4704 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4739 T14 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4763 T14 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 4764 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 4765 T14 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4766 T14 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 4803 T14 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 4846 T14 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 4847 T14 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4880 T14 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 5410 T14 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 5423 T14 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 5427 T14 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5446 T14 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5451 T14 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5456 T14 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5458 T14 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5460 T14 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1352461659291/shard2/data/
[junit4:junit4]   2> 5461 T14 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 5462 T14 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1352461659291/shard2/data/index/
[junit4:junit4]   2> 5462 T14 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1352461659291/shard2/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5463 T14 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1352461659291/shard2/data/index forceNew:false
[junit4:junit4]   2> 5467 T14 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1c5ca6ca lockFactory=org.apache.lucene.store.NativeFSLockFactory@4199d4f9),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5467 T14 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 5469 T14 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 5469 T14 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 5470 T14 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 5470 T14 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 5480 T14 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 5487 T14 oass.SolrIndexSearcher.<init> Opening Searcher@1790423d main
[junit4:junit4]   2> 5487 T14 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 5489 T14 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 5489 T14 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 5490 T14 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 5490 T14 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 5491 T14 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 5491 T14 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 5492 T14 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 5492 T14 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 5493 T14 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 5493 T14 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 5494 T14 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 5494 T14 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 5507 T14 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 5507 T71 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1790423d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5508 T14 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 5508 T14 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 5509 T14 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 5520 T14 oejs.Server.doStart jetty-8.1.7.v20120910
[junit4:junit4]   2> 5522 T14 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:35144
[junit4:junit4]   2> 5524 T14 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 5524 T14 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 5525 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 5573 T14 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 5574 T14 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 5574 T14 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 5575 T14 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 5576 T14 oasc.CoreContainer.<init> New CoreContainer 795252340
[junit4:junit4]   2> 5576 T14 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 5577 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 5612 T14 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 5631 T14 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 5632 T14 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 5634 T14 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 5635 T14 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 5679 T14 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 5733 T14 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 5734 T14 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 5760 T14 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 6329 T14 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 6349 T14 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 6353 T14 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 6372 T14 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6377 T14 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6381 T14 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6383 T14 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6385 T14 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Test

[...truncated too long message...]

hannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   27) Thread[id=71, name=qtp1799456698-71 Acceptor0 SelectChannelConnector@0.0.0.0:19889, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   28) Thread[id=166, name=qtp1514649449-166 Acceptor1 SelectChannelConnector@0.0.0.0:19908, state=BLOCKED, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   29) Thread[id=72, name=qtp1799456698-72 Acceptor1 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   30) Thread[id=141, name=qtp29394673-141 Acceptor3 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   31) Thread[id=86, name=qtp571201483-86 Acceptor1 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   32) Thread[id=85, name=qtp571201483-85 Acceptor0 SelectChannelConnector@0.0.0.0:19892, state=BLOCKED, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   33) Thread[id=52, name=qtp1832089855-52 Acceptor2 SelectChannelConnector@0.0.0.0:19879, state=BLOCKED, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   34) Thread[id=138, name=qtp29394673-138 Acceptor0 SelectChannelConnector@0.0.0.0:19905, state=BLOCKED, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   35) Thread[id=125, name=qtp1017859891-125 Acceptor1 SelectChannelConnector@0.0.0.0:19904, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   36) Thread[id=74, name=qtp1799456698-74 Acceptor3 SelectChannelConnector@0.0.0.0:19889, state=BLOCKED, group=TGRP-TestDistributedGrouping]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([9A74CB360EB6D5D0]:0)
[junit4:junit4] Completed on J1 in 51.15s, 1 test, 2 errors <<< FAILURES!

[...truncated 308 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:335: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/module-build.xml:61: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:1163: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:827: There were test failures: 233 suites, 970 tests, 4 suite-level errors, 872 ignored

Total time: 20 minutes 16 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 615 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/615/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.AnalysisAfterCoreReloadTest

Error Message:
4 threads leaked from SUITE scope at org.apache.solr.AnalysisAfterCoreReloadTest:     1) Thread[id=92, name=qtp1156597185-92 Acceptor3 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=91, name=qtp1156597185-91 Acceptor2 SelectChannelConnector@0.0.0.0:29439, state=RUNNABLE, group=TGRP-AnalysisAfterCoreReloadTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=89, name=qtp1156597185-89 Acceptor0 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=90, name=qtp1156597185-90 Acceptor1 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.AnalysisAfterCoreReloadTest: 
   1) Thread[id=92, name=qtp1156597185-92 Acceptor3 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=91, name=qtp1156597185-91 Acceptor2 SelectChannelConnector@0.0.0.0:29439, state=RUNNABLE, group=TGRP-AnalysisAfterCoreReloadTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=89, name=qtp1156597185-89 Acceptor0 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=90, name=qtp1156597185-90 Acceptor1 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([EFEC412AFD1FA1E8]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.AnalysisAfterCoreReloadTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=92, name=qtp1156597185-92 Acceptor3 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=91, name=qtp1156597185-91 Acceptor2 SelectChannelConnector@0.0.0.0:29439, state=RUNNABLE, group=TGRP-AnalysisAfterCoreReloadTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=89, name=qtp1156597185-89 Acceptor0 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=90, name=qtp1156597185-90 Acceptor1 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=92, name=qtp1156597185-92 Acceptor3 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=91, name=qtp1156597185-91 Acceptor2 SelectChannelConnector@0.0.0.0:29439, state=RUNNABLE, group=TGRP-AnalysisAfterCoreReloadTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=89, name=qtp1156597185-89 Acceptor0 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=90, name=qtp1156597185-90 Acceptor1 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([EFEC412AFD1FA1E8]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.request.TestRemoteStreaming

Error Message:
4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming:     1) Thread[id=25, name=qtp1604840243-25 Acceptor0 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=27, name=qtp1604840243-27 Acceptor2 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=26, name=qtp1604840243-26 Acceptor1 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=28, name=qtp1604840243-28 Acceptor3 SelectChannelConnector@0.0.0.0:29337, state=RUNNABLE, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming: 
   1) Thread[id=25, name=qtp1604840243-25 Acceptor0 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=27, name=qtp1604840243-27 Acceptor2 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=26, name=qtp1604840243-26 Acceptor1 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=28, name=qtp1604840243-28 Acceptor3 SelectChannelConnector@0.0.0.0:29337, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([EFEC412AFD1FA1E8]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.request.TestRemoteStreaming

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=25, name=qtp1604840243-25 Acceptor0 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=27, name=qtp1604840243-27 Acceptor2 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=26, name=qtp1604840243-26 Acceptor1 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=28, name=qtp1604840243-28 Acceptor3 SelectChannelConnector@0.0.0.0:29337, state=RUNNABLE, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=25, name=qtp1604840243-25 Acceptor0 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=27, name=qtp1604840243-27 Acceptor2 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=26, name=qtp1604840243-26 Acceptor1 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=28, name=qtp1604840243-28 Acceptor3 SelectChannelConnector@0.0.0.0:29337, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([EFEC412AFD1FA1E8]:0)




Build Log:
[...truncated 8460 lines...]
[junit4:junit4] Suite: org.apache.solr.request.TestRemoteStreaming
[junit4:junit4]   2> 0 T18 oas.SolrTestCaseJ4.initCore ####initCore
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-TestRemoteStreaming-1352414376210
[junit4:junit4]   2> 1 T18 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4:junit4]   2> 375 T18 oejs.Server.doStart jetty-8.1.7.v20120910
[junit4:junit4]   2> 637 T18 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:29337
[junit4:junit4]   2> 675 T18 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 675 T18 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: solr/
[junit4:junit4]   2> 676 T18 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: 'solr/'
[junit4:junit4]   2> 711 T18 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 712 T18 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 712 T18 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: solr/
[junit4:junit4]   2> 713 T18 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solr/solr.xml
[junit4:junit4]   2> 713 T18 oasc.CoreContainer.<init> New CoreContainer 165061524
[junit4:junit4]   2> 714 T18 oasc.CoreContainer$Initializer.initialize no solr.xml file found - using default
[junit4:junit4]   2> 714 T18 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'solr/'
[junit4:junit4]   2> 715 T18 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'solr/'
[junit4:junit4]   2> 751 T18 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 795 T18 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: solr/collection1
[junit4:junit4]   2> 796 T18 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'solr/collection1/'
[junit4:junit4]   2> 870 T18 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 942 T18 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 946 T18 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 978 T18 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1694 T18 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1713 T18 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1718 T18 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1766 T18 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1787 T18 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1793 T18 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1801 T18 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1805 T18 oasc.SolrCore.<init> [collection1] Opening new SolrCore at solr/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-TestRemoteStreaming-1352414376210/
[junit4:junit4]   2> 1805 T18 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@70317b12
[junit4:junit4]   2> 1808 T18 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-TestRemoteStreaming-1352414376210/index/
[junit4:junit4]   2> 1808 T18 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-TestRemoteStreaming-1352414376210/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1815 T18 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-TestRemoteStreaming-1352414376210/index forceNew:false
[junit4:junit4]   2> 1818 T18 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1cb24b4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@d603051),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1819 T18 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 1820 T18 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 1830 T18 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1831 T18 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 1832 T18 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1833 T18 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 1834 T18 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1834 T18 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1835 T18 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1836 T18 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1837 T18 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1838 T18 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 1839 T18 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 1840 T18 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1841 T18 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 1842 T18 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 1843 T18 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 1844 T18 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1845 T18 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1846 T18 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1847 T18 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1848 T18 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1849 T18 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1850 T18 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1851 T18 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1852 T18 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 1853 T18 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 1866 T18 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1871 T18 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 1873 T18 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: []
[junit4:junit4]   2> 1877 T18 oass.SolrIndexSearcher.<init> Opening Searcher@166fa972 main
[junit4:junit4]   2> 1878 T18 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 1879 T18 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1879 T18 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1880 T18 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 1913 T18 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 1954 T18 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 1954 T18 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 1954 T18 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 1955 T18 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 1955 T18 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 1956 T18 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 1956 T18 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 1956 T18 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 1957 T18 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 1957 T18 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 1973 T33 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@166fa972 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1980 T18 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 1980 T18 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 1981 T18 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 1981 T18 oas.SolrJettyTestBase.createJetty Jetty Assigned Port#29337
[junit4:junit4]   2> 1987 T18 oas.SolrTestCaseJ4.setUp ###Starting testStreamUrl
[junit4:junit4]   2> 2033 T18 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C3 name=collection1 org.apache.solr.core.SolrCore@5c64f59
[junit4:junit4]   2> 2880 T31 C3 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1cb24b4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@d603051),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 2881 T31 C3 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 2883 T31 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1234]} 0 32
[junit4:junit4]   2> 3038 T32 C3 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 3041 T32 C3 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1cb24b4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@d603051),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1cb24b4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@d603051),segFN=segments_2,generation=2,filenames=[_0.tis, _0.nrm, _0.fnm, _0.tii, _0.frq, segments_2, _0.fdx, _0.si, _0.prx, _0.fdt]
[junit4:junit4]   2> 3042 T32 C3 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 3043 T32 C3 oass.SolrIndexSearcher.<init> Opening Searcher@54707929 main
[junit4:junit4]   2> 3044 T32 C3 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3045 T32 C3 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 3049 T33 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@54707929 main{StandardDirectoryReader(segments_2:3 _0(4.1):C1)}
[junit4:junit4]   2> 3050 T32 C3 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 12
[junit4:junit4]   2> 3143 T31 C3 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=60 
[junit4:junit4]   2> 3163 T31 C3 REQ [collection1] webapp=/solr path=/select params={wt=csv&q=*:*&fl=id} hits=1 status=0 QTime=6 
[junit4:junit4]   2> 3205 T32 C3 REQ [collection1] webapp=/solr path=/debug/dump params={wt=xml&stream.url=http://127.0.0.1:29337/solr/select?q%3D*:*%26fl%3Did%26wt%3Dcsv} status=0 QTime=46 
[junit4:junit4]   2> 3213 T18 oas.SolrTestCaseJ4.tearDown ###Ending testStreamUrl
[junit4:junit4]   2> 3251 T18 oas.SolrTestCaseJ4.setUp ###Starting testNoUrlAccess
[junit4:junit4]   2> 3257 T30 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1234]} 0 1
[junit4:junit4]   2> 3261 T32 C3 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 3267 T32 C3 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1cb24b4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@d603051),segFN=segments_2,generation=2,filenames=[_0.tis, _0.nrm, _0.fnm, _0.tii, _0.frq, segments_2, _0.fdx, _0.si, _0.prx, _0.fdt]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1cb24b4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@d603051),segFN=segments_3,generation=3,filenames=[_1.fdx, _1.prx, _1.tis, _1.frq, _1.fdt, _1.si, _1.tii, _1.fnm, _1.nrm, segments_3]
[junit4:junit4]   2> 3267 T32 C3 oasc.SolrDeletionPolicy.updateCommits newest commit = 3
[junit4:junit4]   2> 3270 T32 C3 oass.SolrIndexSearcher.<init> Opening Searcher@299a3d4 main
[junit4:junit4]   2> 3270 T32 C3 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3271 T32 C3 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 3275 T33 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@299a3d4 main{StandardDirectoryReader(segments_3:7 _1(4.1):C1)}
[junit4:junit4]   2> 3275 T32 C3 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 14
[junit4:junit4]   2> 3280 T31 C3 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 3285 T30 C3 REQ [collection1] webapp=/solr path=/select params={wt=javabin&stream.url=http://127.0.0.1:29337/solr/update?commit%3Dtrue%26stream.body%3D%253Cdelete%253E%253Cquery%253E*%253A*%253C%252Fquery%253E%253C%252Fdelete%253E&q=*:*&version=2} hits=1 status=0 QTime=0 
[junit4:junit4]   2> 3289 T32 C3 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 3291 T18 oas.SolrTestCaseJ4.tearDown ###Ending testNoUrlAccess
[junit4:junit4]   2> 3298 T18 oas.SolrTestCaseJ4.setUp ###Starting testMakeDeleteAllUrl
[junit4:junit4]   2> 3304 T31 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1234]} 0 2
[junit4:junit4]   2> 3309 T30 C3 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 3314 T30 C3 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1cb24b4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@d603051),segFN=segments_3,generation=3,filenames=[_1.fdx, _1.prx, _1.tis, _1.frq, _1.fdt, _1.si, _1.tii, _1.fnm, _1.nrm, segments_3]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1cb24b4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@d603051),segFN=segments_4,generation=4,filenames=[_2.si, _2.tis, _2.prx, _2.fdt, _2.frq, _2.tii, _2.fdx, _2.nrm, _2.fnm, segments_4]
[junit4:junit4]   2> 3314 T30 C3 oasc.SolrDeletionPolicy.updateCommits newest commit = 4
[junit4:junit4]   2> 3316 T30 C3 oass.SolrIndexSearcher.<init> Opening Searcher@6df8952f main
[junit4:junit4]   2> 3316 T30 C3 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3317 T30 C3 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 3321 T33 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6df8952f main{StandardDirectoryReader(segments_4:11 _2(4.1):C1)}
[junit4:junit4]   2> 3322 T30 C3 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 13
[junit4:junit4]   2> 3326 T32 C3 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=0 
[junit4:junit4]   2> 3330 T31 C3 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4:junit4]   2> 3331 T31 C3 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 3332 T31 C3 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1cb24b4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@d603051),segFN=segments_4,generation=4,filenames=[_2.si, _2.tis, _2.prx, _2.fdt, _2.frq, _2.tii, _2.fdx, _2.nrm, _2.fnm, segments_4]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1cb24b4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@d603051),segFN=segments_5,generation=5,filenames=[segments_5]
[junit4:junit4]   2> 3333 T31 C3 oasc.SolrDeletionPolicy.updateCommits newest commit = 5
[junit4:junit4]   2> 3334 T31 C3 oass.SolrIndexSearcher.<init> Opening Searcher@47ffa006 main
[junit4:junit4]   2> 3335 T31 C3 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3336 T31 C3 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 3338 T33 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@47ffa006 main{StandardDirectoryReader(segments_5:12)}
[junit4:junit4]   2> 3339 T31 C3 UPDATE [collection1] webapp=/solr path=/update params={commit=true&stream.body=<delete><query>*:*</query></delete>} {deleteByQuery=*:*,commit=} 0 10
[junit4:junit4]   2> 3344 T30 C3 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 3345 T18 oas.SolrTestCaseJ4.tearDown ###Ending testMakeDeleteAllUrl
[junit4:junit4]   2> 3353 T18 oas.SolrTestCaseJ4.setUp ###Starting testQtUpdateFails
[junit4:junit4]   2> 3358 T32 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1234]} 0 2
[junit4:junit4]   2> 3364 T31 C3 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 3366 T31 C3 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1cb24b4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@d603051),segFN=segments_5,generation=5,filenames=[segments_5]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1cb24b4b lockFactory=org.apache.lucene.store.NativeFSLockFactory@d603051),segFN=segments_6,generation=6,filenames=[_3.fdx, _3.nrm, _3.tii, _3.frq, _3.si, _3.fdt, _3.prx, _3.fnm, segments_6, _3.tis]
[junit4:junit4]   2> 3367 T31 C3 oasc.SolrDeletionPolicy.updateCommits newest commit = 6
[junit4:junit4]   2> 3419 T31 C3 oass.SolrIndexSearcher.<init> Opening Searcher@67588300 main
[junit4:junit4]   2> 3419 T31 C3 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3420 T31 C3 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 3423 T33 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@67588300 main{StandardDirectoryReader(segments_6:14 _3(4.1):C1)}
[junit4:junit4]   2> 3424 T31 C3 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 60
[junit4:junit4]   2> 3428 T30 C3 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 3433 T18 oas.SolrTestCaseJ4.tearDown ###Ending testQtUpdateFails
[junit4:junit4]   2> 3470 T18 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=165061524
[junit4:junit4]   2> 3471 T18 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@5c64f59
[junit4:junit4]   2> 3486 T18 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=5,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=4,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4]   2> 3487 T18 oasc.SolrCore.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 3487 T18 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 3487 T18 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 3489 T18 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 3495 T18 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 3596 T18 oejut.QueuedThreadPool.doStop WARNING 4 threads could not be stopped
[junit4:junit4]   2> 3596 T18 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> 3607 T17 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 4 leaked thread(s).
[junit4:junit4]   2> 23715 T17 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming: 
[junit4:junit4]   2> 	   1) Thread[id=25, name=qtp1604840243-25 Acceptor0 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   2) Thread[id=27, name=qtp1604840243-27 Acceptor2 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   3) Thread[id=26, name=qtp1604840243-26 Acceptor1 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   4) Thread[id=28, name=qtp1604840243-28 Acceptor3 SelectChannelConnector@0.0.0.0:29337, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 23716 T17 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
[junit4:junit4]   2> 	   1) Thread[id=25, name=qtp1604840243-25 Acceptor0 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	   2) Thread[id=27, name=qtp1604840243-27 Acceptor2 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	   3) Thread[id=26, name=qtp1604840243-26 Acceptor1 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	   4) Thread[id=28, name=qtp1604840243-28 Acceptor3 SelectChannelConnector@0.0.0.0:29337, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 26761 T17 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
[junit4:junit4]   2> 	   1) Thread[id=25, name=qtp1604840243-25 Acceptor0 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   2) Thread[id=27, name=qtp1604840243-27 Acceptor2 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   3) Thread[id=26, name=qtp1604840243-26 Acceptor1 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   4) Thread[id=28, name=qtp1604840243-28 Acceptor3 SelectChannelConnector@0.0.0.0:29337, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene3x, sim=DefaultSimilarity, locale=es_DO, timezone=Europe/Belgrade
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=5,free=62022968,total=130023424
[junit4:junit4]   2> NOTE: All tests run in this JVM: [SuggesterTSTTest, BasicFunctionalityTest, TestRemoteStreaming]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestRemoteStreaming -Dtests.seed=EFEC412AFD1FA1E8 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_DO -Dtests.timezone=Europe/Belgrade -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR   0.00s J1 | TestRemoteStreaming (suite) <<<
[junit4:junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming: 
[junit4:junit4]    >    1) Thread[id=25, name=qtp1604840243-25 Acceptor0 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=27, name=qtp1604840243-27 Acceptor2 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=26, name=qtp1604840243-26 Acceptor1 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=28, name=qtp1604840243-28 Acceptor3 SelectChannelConnector@0.0.0.0:29337, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([EFEC412AFD1FA1E8]:0)
[junit4:junit4]    > Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4]    >    1) Thread[id=25, name=qtp1604840243-25 Acceptor0 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=27, name=qtp1604840243-27 Acceptor2 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=26, name=qtp1604840243-26 Acceptor1 SelectChannelConnector@0.0.0.0:29337, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=28, name=qtp1604840243-28 Acceptor3 SelectChannelConnector@0.0.0.0:29337, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([EFEC412AFD1FA1E8]:0)
[junit4:junit4] Completed on J1 in 26.91s, 4 tests, 2 errors <<< FAILURES!

[...truncated 523 lines...]
[junit4:junit4] Suite: org.apache.solr.AnalysisAfterCoreReloadTest
[junit4:junit4]   2> 5163 T81 oasu.AbstractSolrTestCase.setUp ####SETUP_START testStopwordsAfterCoreReload
[junit4:junit4]   2> 5165 T81 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1/'
[junit4:junit4]   2> 5167 T81 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solr-test-home-21381301548538531/collection1/lib/README' to classloader
[junit4:junit4]   2> 5168 T81 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solr-test-home-21381301548538531/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 5260 T81 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 5331 T81 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 5332 T81 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 5341 T81 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 5983 T81 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 5995 T81 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 5999 T81 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 6014 T81 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6019 T81 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6024 T81 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6025 T81 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6026 T81 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 6026 T81 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531
[junit4:junit4]   2> 6027 T81 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/'
[junit4:junit4]   2> 6039 T81 oasc.CoreContainer.<init> New CoreContainer 75451581
[junit4:junit4]   2> 6039 T81 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.AnalysisAfterCoreReloadTest-1352414417679/
[junit4:junit4]   2> 6040 T81 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@682efa99
[junit4:junit4]   2> 6041 T81 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.AnalysisAfterCoreReloadTest-1352414417679/index/
[junit4:junit4]   2> 6041 T81 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.AnalysisAfterCoreReloadTest-1352414417679/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 6043 T81 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.AnalysisAfterCoreReloadTest-1352414417679/index forceNew:false
[junit4:junit4]   2> 6046 T81 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5fc52571 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3c22c7c3),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 6047 T81 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 6048 T81 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 6051 T81 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 6052 T81 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 6052 T81 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 6053 T81 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 6054 T81 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 6055 T81 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 6055 T81 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 6056 T81 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 6057 T81 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 6058 T81 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 6059 T81 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 6060 T81 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 6060 T81 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 6061 T81 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 6061 T81 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 6062 T81 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 6062 T81 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6063 T81 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6064 T81 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6064 T81 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6065 T81 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6065 T81 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6066 T81 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6067 T81 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 6068 T81 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 6078 T81 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 6081 T81 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 6083 T81 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [THROW.ERROR.ON.ADD.UPDATEPROCESSOR.JS, SOLRCONFIG-HIGHLIGHT.XML, SCHEMA-REQUIRED-FIELDS.XML, SCHEMA-REPLICATION2.XML, TRIVIAL.UPDATEPROCESSOR1.JS, SCHEMA-MINIMAL.XML, BAD-SCHEMA-DUP-DYNAMICFIELD.XML, SOLRCONFIG-CACHING.XML, SOLRCONFIG-REPEATER.XML, BAD-SCHEMA-NONTEXT-ANALYZER.XML, CURRENCY.XML, SOLRCONFIG-MERGEPOLICY.XML, SOLRCONFIG-TLOG.XML, SOLRCONFIG-MASTER.XML, BAD-SCHEMA-UNIQUEKEY-MULTIVALUED.XML, SCHEMA11.XML, SOLRCONFIG-BASIC.XML, DA_COMPOUNDDICTIONARY.TXT, MISSLEADING.EXTENSION.UPDATEPROCESSOR.JS.TXT, SCHEMA-COPYFIELD-TEST.XML, SOLRCONFIG-SLAVE.XML, SOLRCONFIG-PROPINJECT-INDEXDEFAULT.XML, ELEVATE.XML, TRIVIAL.UPDATEPROCESSOR0.JS, BAD-SOLRCONFIG-BOGUS-SCRIPTENGINE-NAME.XML, SCHEMA-IB.XML, SCHEMA-CHARFILTERS.XML, SOLRCONFIG-QUERYSENDER.XML, DA_UTF8.XML, SCHEMA-REPLICATION1.XML, SOLRCONFIG-SNIPPET-PROCESSOR.XML, CONDITIONAL.UPDATEPROCESSOR.JS, MISSING.FUNCTIONS.UPDATEPROCESSOR.JS, SOLRCONFIG-ENABLEPLUGIN.XML, HYPHENATION.DTD, STEMDICT.TXT, SCHEMA-PHRASESUGGEST.XML, HUNSPELL-TEST.AFF, SCHEMA-SNIPPET-TYPE.XML, STOPTYPES-1.TXT, STOPWORDSWRONGENCODING.TXT, SCHEMA-NUMERIC.XML, SOLRCONFIG-TRANSFORMERS.XML, SOLRCONFIG-PROPINJECT.XML, BAD-SCHEMA-NOT-INDEXED-BUT-TF.XML, SOLRCONFIG-SIMPLELOCK.XML, WDFTYPES.TXT, SCHEMA-REVERSED.XML, STOPTYPES-2.TXT, BAD-SCHEMA-CURRENCY-FT-MULTIVALUED.XML, SOLRCONFIG-SPELLCHECKCOMPONENT.XML, BAD-SCHEMA-CURRENCY-MULTIVALUED.XML, SCHEMA-DFR.XML, SOLRCONFIG-PHRASESUGGEST.XML, BAD-SCHEMA-NOT-INDEXED-BUT-POS.XML, KEEP-1.TXT, OPEN-EXCHANGE-RATES.JSON, SCHEMA-SPATIAL.XML, STOPWITHBOM.TXT, SOLRCONFIG-SPELLCHECKER.XML, SCHEMA-BINARYFIELD.XML, SOLRCONFIG-UPDATE-PROCESSOR-CHAINS.XML, BAD-SCHEMA-OMIT-TF-BUT-NOT-POS.XML, SCHEMA-XINCLUDE.XML, BAD-SCHEMA-DUP-FIELDTYPE.XML, SOLRCONFIG-MASTER1.XML, SYNONYMS.TXT, BAD-SCHEMA-CURRENCY-DYNAMIC-MULTIVALUED.XML, SCHEMA_CODEC.XML, SCHEMA.XML, SOLRCONFIG-SOLR-749.XML, SOLRCONFIG-MASTER1-KEEPONEBACKUP.XML, STOP-2.TXT, SOLRCONFIG-FUNCTIONQUERY.XML, SCHEMA-LMDIRICHLET.XML, SOLRCONFIG-TERMINDEX.XML, SOLRCONFIG-ELEVATE.XML, STOPWORDS.TXT, SCHEMA-FOLDING.XML, SCHEMA-STOP-KEEP.XML, BAD-SCHEMA-NOT-INDEXED-BUT-NORMS.XML, SOLRCONFIG-SOLCOREPROPERTIES.XML, STOP-1.TXT, SOLRCONFIG-MASTER2.XML, SCHEMA-SPELLCHECKER.XML, SOLRCONFIG-RESPONSE-LOG-COMPONENT.XML, SOLRCONFIG-LAZYWRITER.XML, SCHEMA-LUCENEMATCHVERSION.XML, BAD-MP-SOLRCONFIG.XML, FRENCHARTICLES.TXT, SCHEMA15.XML, SOLRCONFIG-REQHANDLER.INCL, SCHEMASURROUND.XML, SCHEMA-COLLATEFILTER.XML, SOLRCONFIG-MASTER3.XML, HUNSPELL-TEST.DIC, BAD-SCHEMA-CODEC-GLOBAL-VS-FT-MISMATCH.XML, SOLRCONFIG-XINCLUDE.XML, SOLRCONFIG-DELPOLICY1.XML, SOLRCONFIG-SLAVE1.XML, SCHEMA-SIM.XML, SCHEMA-SNIPPET-FIELD.XML, SCHEMA-COLLATE.XML, STOP-SNOWBALL.TXT, BAD-SCHEMA-SIM-GLOBAL-VS-FT-MISMATCH.XML, PROTWORDS.TXT, SCHEMA-TRIE.XML, SOLRCONFIG_CODEC.XML, BAD-SOLRCONFIG-INVALID-SCRIPTFILE.XML, JASUGGEST.TXT, SCHEMA-TFIDF.XML, SOLRCONFIG-SCRIPT-UPDATEPROCESSOR.XML, SCHEMA-LMJELINEKMERCER.XML, PHRASESUGGEST.TXT, BAD-SOLRCONFIG-MISSING-SCRIPTFILE.XML, SOLRCONFIG-BASIC-LUCENEVERSION31.XML, BAD-SCHEMA-UNIQUEKEY-USES-DEFAULT.XML, OLD_SYNONYMS.TXT, SOLRCONFIG-DELPOLICY2.XML, XSLT, SOLRCONFIG-NATIVELOCK.XML, BAD-SCHEMA-DUP-FIELD.XML, SOLRCONFIG-NOCACHE.XML, SCHEMA-BM25.XML, ADDFIELDS.UPDATEPROCESSOR.JS, SOLRCONFIG-QUERYSENDER-NOQUERY.XML, SOLRCONFIG-ALTDIRECTORY.XML, COMPOUNDDICTIONARY.TXT, SOLRCONFIG-INDEXCONFIG.XML, SOLRCONFIG_PERF.XML, BAD-SCHEMA-ANALYZER-CLASS-AND-NESTED.XML, SCHEMA-NOT-REQUIRED-UNIQUE-KEY.XML, KEEP-2.TXT, BAD-SCHEMA-UNIQUEKEY-IS-COPYFIELD-DEST.XML, SCHEMA12.XML, MAPPING-ISOLATIN1ACCENT.TXT, BAD_SOLRCONFIG.XML, BAD-SCHEMA-EXTERNAL-FILEFIELD.XML]
[junit4:junit4]   2> 6085 T81 oass.SolrIndexSearcher.<init> Opening Searcher@1dd0a8c0 main
[junit4:junit4]   2> 6085 T81 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 6086 T81 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 6086 T81 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 6086 T81 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 6097 T81 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 6147 T81 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 6148 T81 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 6148 T81 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 6148 T81 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 6149 T81 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 6149 T81 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 6149 T81 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 6149 T81 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 6150 T81 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 6150 T81 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 6163 T82 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1dd0a8c0 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 6166 T81 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 6175 T81 oasu.AbstractSolrTestCase.setUp ####SETUP_END testStopwordsAfterCoreReload
[junit4:junit4]   2> 6569 T81 oejs.Server.doStart jetty-8.1.7.v20120910
[junit4:junit4]   2> 6827 T81 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:29439
[junit4:junit4]   2> 6864 T81 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 6865 T81 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531
[junit4:junit4]   2> 6865 T81 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/'
[junit4:junit4]   2> 6898 T81 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 6898 T81 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 6899 T81 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531
[junit4:junit4]   2> 6899 T81 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/solr.xml
[junit4:junit4]   2> 6899 T81 oasc.CoreContainer.<init> New CoreContainer 1932761421
[junit4:junit4]   2> 6900 T81 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/'
[junit4:junit4]   2> 6900 T81 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/'
[junit4:junit4]   2> 6924 T81 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 6940 T81 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1
[junit4:junit4]   2> 6940 T81 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1/'
[junit4:junit4]   2> 6941 T81 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solr-test-home-21381301548538531/collection1/lib/README' to classloader
[junit4:junit4]   2> 6942 T81 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solr-test-home-21381301548538531/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 6979 T81 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 7028 T81 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 7029 T81 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 7035 T81 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 7508 T81 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 7517 T81 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 7520 T81 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 7530 T81 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 7534 T81 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 7538 T81 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 7538 T81 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 7539 T81 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1/data/
[junit4:junit4]   2> 7540 T81 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@682efa99
[junit4:junit4]   2> 7541 T81 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1/data/index/
[junit4:junit4]   2> 7541 T81 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 7542 T81 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1/data/index forceNew:false
[junit4:junit4]   2> 7543 T81 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@680e6039 lockFactory=org.apache.lucene.store.NativeFSLockFactory@60e5272b; maxCacheMB=0.8359375 maxMergeSizeMB=0.57421875))),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 7544 T81 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 7544 T81 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 7547 T81 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 7547 T81 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 7547 T81 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 7548 T81 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 7548 T81 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 7549 T81 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 7549 T81 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 7550 T81 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 7550 T81 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 7551 T81 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 7552 T81 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 7552 T81 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 7553 T81 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 7553 T81 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 7553 T81 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 7554 T81 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 7554 T81 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7555 T81 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7555 T81 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7556 T81 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7556 T81 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7557 T81 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7558 T81 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7558 T81 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 7559 T81 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 7567 T81 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 7570 T81 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 7572 T81 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [THROW.ERROR.ON.ADD.UPDATEPROCESSOR.JS, SOLRCONFIG-HIGHLIGHT.XML, SCHEMA-REQUIRED-FIELDS.XML, SCHEMA-REPLICATION2.XML, TRIVIAL.UPDATEPROCESSOR1.JS, SCHEMA-MINIMAL.XML, BAD-SCHEMA-DUP-DYNAMICFIELD.XML, SOLRCONFIG-CACHING.XML, SOLRCONFIG-REPEATER.XML, BAD-SCHEMA-NONTEXT-ANALYZER.XML, CURRENCY.XML, SOLRCONFIG-MERGEPOLICY.XML, SOLRCONFIG-TLOG.XML, SOLRCONFIG-MASTER.XML, BAD-SCHEMA-UNIQUEKEY-MULTIVALUED.XML, SCHEMA11.XML, SOLRCONFIG-BASIC.XML, DA_COMPOUNDDICTIONARY.TXT, MISSLEADING.EXTENSION.UPDATEPROCESSOR.JS.TXT, SCHEMA-COPYFIELD-TEST.XML, SOLRCONFIG-SLAVE.XML, SOLRCONFIG-PROPINJECT-INDEXDEFAULT.XML, ELEVATE.XML, TRIVIAL.UPDATEPROCESSOR0.JS, BAD-SOLRCONFIG-BOGUS-SCRIPTENGINE-NAME.XML, SCHEMA-IB.XML, SCHEMA-CHARFILTERS.XML, SOLRCONFIG-QUERYSENDER.XML, DA_UTF8.XML, SCHEMA-REPLICATION1.XML, SOLRCONFIG-SNIPPET-PROCESSOR.XML, CONDITIONAL.UPDATEPROCESSOR.JS, MISSING.FUNCTIONS.UPDATEPROCESSOR.JS, SOLRCONFIG-ENABLEPLUGIN.XML, HYPHENATION.DTD, STEMDICT.TXT, SCHEMA-PHRASESUGGEST.XML, HUNSPELL-TEST.AFF, SCHEMA-SNIPPET-TYPE.XML, STOPTYPES-1.TXT, STOPWORDSWRONGENCODING.TXT, SCHEMA-NUMERIC.XML, SOLRCONFIG-TRANSFORMERS.XML, SOLRCONFIG-PROPINJECT.XML, BAD-SCHEMA-NOT-INDEXED-BUT-TF.XML, SOLRCONFIG-SIMPLELOCK.XML, WDFTYPES.TXT, SCHEMA-REVERSED.XML, STOPTYPES-2.TXT, BAD-SCHEMA-CURRENCY-FT-MULTIVALUED.XML, SOLRCONFIG-SPELLCHECKCOMPONENT.XML, BAD-SCHEMA-CURRENCY-MULTIVALUED.XML, SCHEMA-DFR.XML, SOLRCONFIG-PHRASESUGGEST.XML, BAD-SCHEMA-NOT-INDEXED-BUT-POS.XML, KEEP-1.TXT, OPEN-EXCHANGE-RATES.JSON, SCHEMA-SPATIAL.XML, STOPWITHBOM.TXT, SOLRCONFIG-SPELLCHECKER.XML, SCHEMA-BINARYFIELD.XML, SOLRCONFIG-UPDATE-PROCESSOR-CHAINS.XML, BAD-SCHEMA-OMIT-TF-BUT-NOT-POS.XML, SCHEMA-XINCLUDE.XML, BAD-SCHEMA-DUP-FIELDTYPE.XML, SOLRCONFIG-MASTER1.XML, SYNONYMS.TXT, BAD-SCHEMA-CURRENCY-DYNAMIC-MULTIVALUED.XML, SCHEMA_CODEC.XML, SCHEMA.XML, SOLRCONFIG-SOLR-749.XML, SOLRCONFIG-MASTER1-KEEPONEBACKUP.XML, STOP-2.TXT, SOLRCONFIG-FUNCTIONQUERY.XML, SCHEMA-LMDIRICHLET.XML, SOLRCONFIG-TERMINDEX.XML, SOLRCONFIG-ELEVATE.XML, STOPWORDS.TXT, SCHEMA-FOLDING.XML, SCHEMA-STOP-KEEP.XML, BAD-SCHEMA-NOT-INDEXED-BUT-NORMS.XML, SOLRCONFIG-SOLCOREPROPERTIES.XML, STOP-1.TXT, SOLRCONFIG-MASTER2.XML, SCHEMA-SPELLCHECKER.XML, SOLRCONFIG-RESPONSE-LOG-COMPONENT.XML, SOLRCONFIG-LAZYWRITER.XML, SCHEMA-LUCENEMATCHVERSION.XML, BAD-MP-SOLRCONFIG.XML, FRENCHARTICLES.TXT, SCHEMA15.XML, SOLRCONFIG-REQHANDLER.INCL, SCHEMASURROUND.XML, SCHEMA-COLLATEFILTER.XML, SOLRCONFIG-MASTER3.XML, HUNSPELL-TEST.DIC, BAD-SCHEMA-CODEC-GLOBAL-VS-FT-MISMATCH.XML, SOLRCONFIG-XINCLUDE.XML, SOLRCONFIG-DELPOLICY1.XML, SOLRCONFIG-SLAVE1.XML, SCHEMA-SIM.XML, SCHEMA-SNIPPET-FIELD.XML, SCHEMA-COLLATE.XML, STOP-SNOWBALL.TXT, BAD-SCHEMA-SIM-GLOBAL-VS-FT-MISMATCH.XML, PROTWORDS.TXT, SCHEMA-TRIE.XML, SOLRCONFIG_CODEC.XML, BAD-SOLRCONFIG-INVALID-SCRIPTFILE.XML, JASUGGEST.TXT, SCHEMA-TFIDF.XML, SOLRCONFIG-SCRIPT-UPDATEPROCESSOR.XML, SCHEMA-LMJELINEKMERCER.XML, PHRASESUGGEST.TXT, BAD-SOLRCONFIG-MISSING-SCRIPTFILE.XML, SOLRCONFIG-BASIC-LUCENEVERSION31.XML, BAD-SCHEMA-UNIQUEKEY-USES-DEFAULT.XML, OLD_SYNONYMS.TXT, SOLRCONFIG-DELPOLICY2.XML, XSLT, SOLRCONFIG-NATIVELOCK.XML, BAD-SCHEMA-DUP-FIELD.XML, SOLRCONFIG-NOCACHE.XML, SCHEMA-BM25.XML, ADDFIELDS.UPDATEPROCESSOR.JS, SOLRCONFIG-QUERYSENDER-NOQUERY.XML, SOLRCONFIG-ALTDIRECTORY.XML, COMPOUNDDICTIONARY.TXT, SOLRCONFIG-INDEXCONFIG.XML, SOLRCONFIG_PERF.XML, BAD-SCHEMA-ANALYZER-CLASS-AND-NESTED.XML, SCHEMA-NOT-REQUIRED-UNIQUE-KEY.XML, KEEP-2.TXT, BAD-SCHEMA-UNIQUEKEY-IS-COPYFIELD-DEST.XML, SCHEMA12.XML, MAPPING-ISOLATIN1ACCENT.TXT, BAD_SOLRCONFIG.XML, BAD-SCHEMA-EXTERNAL-FILEFIELD.XML]
[junit4:junit4]   2> 7574 T81 oass.SolrIndexSearcher.<init> Opening Searcher@19ccda74 main
[junit4:junit4]   2> 7575 T81 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 7575 T81 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 7575 T81 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 7576 T81 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 7578 T81 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 7588 T81 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 7588 T81 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 7589 T81 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 7589 T81 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 7589 T81 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 7590 T81 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 7590 T81 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 7590 T81 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 7591 T81 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 7591 T81 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 7605 T97 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@19ccda74 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 7610 T81 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 7611 T81 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 7611 T81 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 7696 T81 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C14 name=collection1 org.apache.solr.core.SolrCore@3d88eb46
[junit4:junit4]   2> 8668 T96 C14 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@680e6039 lockFactory=org.apache.lucene.store.NativeFSLockFactory@60e5272b; maxCacheMB=0.8359375 maxMergeSizeMB=0.57421875))),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 8669 T96 C14 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 8671 T96 C14 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 8719 T96 C14 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@680e6039 lockFactory=org.apache.lucene.store.NativeFSLockFactory@60e5272b; maxCacheMB=0.8359375 maxMergeSizeMB=0.57421875))),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@680e6039 lockFactory=org.apache.lucene.store.NativeFSLockFactory@60e5272b; maxCacheMB=0.8359375 maxMergeSizeMB=0.57421875))),segFN=segments_2,generation=2,filenames=[_0.fnm, _0_TestBloomFilteredLucene41Postings_0.blm, _0_MockSep_0.pos, _0_Lucene41_0.pos, _0_TestBloomFilteredLucene41Postings_0.tim, _0_nrm.cfs, _0_TestBloomFilteredLucene41Postings_0.tip, _0_TestBloomFilteredLucene41Postings_0.pos, _0_MockSep_0.tib, _0_Lucene41_0.doc, _0_MockSep_0.skp, _0_nrm.cfe, _0_TestBloomFilteredLucene41Postings_0.doc, _0_MockSep_0.frq, segments_2, _0.fdx, _0_MockSep_0.tii, _0.si, _0_MockSep_0.pyl, _0_MockSep_0.doc, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip]
[junit4:junit4]   2> 8720 T96 C14 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 8778 T96 C14 oass.SolrIndexSearcher.<init> Opening Searcher@6cb320ac main
[junit4:junit4]   2> 8779 T96 C14 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 8780 T96 C14 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 8781 T97 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6cb320ac main{StandardDirectoryReader(segments_2:3 _0(4.1):C1)}
[junit4:junit4]   2> 8782 T96 C14 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {add=[42],commit=} 0 140
[junit4:junit4]   2> 8921 T81 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9025 T94 C14 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=teststop:terma&version=2} hits=1 status=0 QTime=57 
[junit4:junit4]   2> 9027 T81 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9039 T95 C14 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=teststop:stopworda&version=2} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 9040 T81 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9051 T96 C14 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=teststop:stopwordb&version=2} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 9051 T81 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9060 T95 C14 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=teststop:stopwordc&version=2} hits=1 status=0 QTime=0 
[junit4:junit4]   2> 9062 T81 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9081 T96 oasc.CoreContainer.reload Reloading SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1
[junit4:junit4]   2> 9082 T96 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1/'
[junit4:junit4]   2> 9083 T96 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solr-test-home-21381301548538531/collection1/lib/README' to classloader
[junit4:junit4]   2> 9083 T96 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solr-test-home-21381301548538531/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 9138 T96 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 9187 T96 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 9188 T96 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 9194 T96 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 9700 T96 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 9718 T96 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 9721 T96 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 9731 T96 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9734 T96 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9737 T96 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9738 T96 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9739 T96 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1/data/
[junit4:junit4]   2> 9740 T96 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@682efa99
[junit4:junit4]   2> 9741 T96 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solr-test-home-21381301548538531/collection1/data/index/
[junit4:junit4]   2> 9741 T96 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 9743 T96 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 9743 T96 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 9744 T96 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 9744 T96 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 9745 T96 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 9745 T96 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 9745 T96 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 9746 T96 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 9746 T96 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 9747 T96 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 9747 T96 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 9748 T96 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 9748 T96 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 9748 T96 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 9749 T96 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 9749 T96 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 9750 T96 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9750 T96 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9751 T96 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9751 T96 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9751 T96 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9752 T96 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9752 T96 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9753 T96 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 9753 T96 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 9762 T96 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 9763 T96 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 9764 T96 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [THROW.ERROR.ON.ADD.UPDATEPROCESSOR.JS, SOLRCONFIG-HIGHLIGHT.XML, SCHEMA-REQUIRED-FIELDS.XML, SCHEMA-REPLICATION2.XML, TRIVIAL.UPDATEPROCESSOR1.JS, SCHEMA-MINIMAL.XML, BAD-SCHEMA-DUP-DYNAMICFIELD.XML, SOLRCONFIG-CACHING.XML, SOLRCONFIG-REPEATER.XML, BAD-SCHEMA-NONTEXT-ANALYZER.XML, CURRENCY.XML, SOLRCONFIG-MERGEPOLICY.XML, SOLRCONFIG-TLOG.XML, SOLRCONFIG-MASTER.XML, BAD-SCHEMA-UNIQUEKEY-MULTIVALUED.XML, SCHEMA11.XML, SOLRCONFIG-BASIC.XML, DA_COMPOUNDDICTIONARY.TXT, MISSLEADING.EXTENSION.UPDATEPROCESSOR.JS.TXT, SCHEMA-COPYFIELD-TEST.XML, SOLRCONFIG-SLAVE.XML, SOLRCONFIG-PROPINJECT-INDEXDEFAULT.XML, ELEVATE.XML, TRIVIAL.UPDATEPROCESSOR0.JS, BAD-SOLRCONFIG-BOGUS-SCRIPTENGINE-NAME.XML, SCHEMA-IB.XML, SCHEMA-CHARFILTERS.XML, SOLRCONFIG-QUERYSENDER.XML, DA_UTF8.XML, SCHEMA-REPLICATION1.XML, SOLRCONFIG-SNIPPET-PROCESSOR.XML, CONDITIONAL.UPDATEPROCESSOR.JS, MISSING.FUNCTIONS.UPDATEPROCESSOR.JS, SOLRCONFIG-ENABLEPLUGIN.XML, HYPHENATION.DTD, STEMDICT.TXT, SCHEMA-PHRASESUGGEST.XML, HUNSPELL-TEST.AFF, SCHEMA-SNIPPET-TYPE.XML, STOPTYPES-1.TXT, STOPWORDSWRONGENCODING.TXT, SCHEMA-NUMERIC.XML, SOLRCONFIG-TRANSFORMERS.XML, SOLRCONFIG-PROPINJECT.XML, BAD-SCHEMA-NOT-INDEXED-BUT-TF.XML, SOLRCONFIG-SIMPLELOCK.XML, WDFTYPES.TXT, SCHEMA-REVERSED.XML, STOPTYPES-2.TXT, BAD-SCHEMA-CURRENCY-FT-MULTIVALUED.XML, SOLRCONFIG-SPELLCHECKCOMPONENT.XML, BAD-SCHEMA-CURRENCY-MULTIVALUED.XML, SCHEMA-DFR.XML, SOLRCONFIG-PHRASESUGGEST.XML, BAD-SCHEMA-NOT-INDEXED-BUT-POS.XML, KEEP-1.TXT, OPEN-EXCHANGE-RATES.JSON, SCHEMA-SPATIAL.XML, STOPWITHBOM.TXT, SOLRCONFIG-SPELLCHECKER.XML, SCHEMA-BINARYFIELD.XML, SOLRCONFIG-UPDATE-PROCESSOR-CHAINS.XML, BAD-SCHEMA-OMIT-TF-BUT-NOT-POS.XML, SCHEMA-XINCLUDE.XML, BAD-SCHEMA-DUP-FIELDTYPE.XML, SOLRCONFIG-MASTER1.XML, SYNONYMS.TXT, BAD-SCHEMA-CURRENCY-DYNAMIC-MULTIVALUED.XML, SCHEMA_CODEC.XML, SCHEMA.XML, SOLRCONFIG-SOLR-749.XML, SOLRCONFIG-MASTER1-KEEPONEBACKUP.XML, STOP-2.TXT, SOLRCONFIG-FUNCTIONQUERY.XML, SCHEMA-LMDIRICHLET.XML, SOLRCONFIG-TERMINDEX.XML, SOLRCONFIG-ELEVATE.XML, STOPWORDS.TXT, SCHEMA-FOLDING.XML, SCHEMA-STOP-KEEP.XML, BAD-SCHEMA-NOT-INDEXED-BUT-NORMS.XML, SOLRCONFIG-SOLCOREPROPERTIES.XML, STOP-1.TXT, SOLRCONFIG-MASTER2.XML, SCHEMA-SPELLCHECKER.XML, SOLRCONFIG-RESPONSE-LOG-COMPONENT.XML, SOLRCONFIG-LAZYWRITER.XML, SCHEMA-LUCENEMATCHVERSION.XML, BAD-MP-SOLRCONFIG.XML, FRENCHARTICLES.TXT, SCHEMA15.XML, SOLRCONFIG-REQHANDLER.INCL, SCHEMASURROUND.XML, SCHEMA-COLLATEFILTER.XML, SOLRCONFIG-MASTER3.XML, HUNSPELL-TEST.DIC, BAD-SCHEMA-CODEC-GLOBAL-VS-FT-MISMATCH.XML, SOLRCONFIG-XINCLUDE.XML, SOLRCONFIG-DELPOLICY1.XML, SOLRCONFIG-SLAVE1.XML, SCHEMA-SIM.XML, SCHEMA-SNIPPET-FIELD.XML, SCHEMA-COLLATE.XML, STOP-SNOWBALL.TXT, BAD-SCHEMA-SIM-GLOBAL-VS-FT-MISMATCH.XML, PROTWORDS.TXT, SCHEMA-TRIE.XML, SOLRCONFIG_CODEC.XML, BAD-SOLRCONFIG-INVALID-SCRIPTFILE.XML, JASUGGEST.TXT, SCHEMA-TFIDF.XML, SOLRCONFIG-SCRIPT-UPDATEPROCESSOR.XML, SCHEMA-LMJELINEKMERCER.XML, PHRASESUGGEST.TXT, BAD-SOLRCONFIG-MISSING-SCRIPTFILE.XML, SOLRCONFIG-BASIC-LUCENEVERSION31.XML, BAD-SCHEMA-UNIQUEKEY-USES-DEFAULT.XML, OLD_SYNONYMS.TXT, SOLRCONFIG-DELPOLICY2.XML, XSLT, SOLRCONFIG-NATIVELOCK.XML, BAD-SCHEMA-DUP-FIELD.XML, SOLRCONFIG-NOCACHE.XML, SCHEMA-BM25.XML, ADDFIELDS.UPDATEPROCESSOR.JS, SOLRCONFIG-QUERYSENDER-NOQUERY.XML, SOLRCONFIG-ALTDIRECTORY.XML, COMPOUNDDICTIONARY.TXT, SOLRCONFIG-INDEXCONFIG.XML, SOLRCONFIG_PERF.XML, BAD-SCHEMA-ANALYZER-CLASS-AND-NESTED.XML, SCHEMA-NOT-REQUIRED-UNIQUE-KEY.XML, KEEP-2.TXT, BAD-SCHEMA-UNIQUEKEY-IS-COPYFIELD-DEST.XML, SCHEMA12.XML, MAPPING-ISOLATIN1ACCENT.TXT, BAD_SOLRCONFIG.XML, BAD-SCHEMA-EXTERNAL-FILEFIELD.XML]
[junit4:junit4]   2> 9771 T96 oass.SolrIndexSearcher.<init> Opening Searcher@5cd76582 main
[junit4:junit4]   2> 9771 T96 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 9772 T96 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 9772 T96 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 9772 T96 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 9774 T96 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 9788 T98 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5cd76582 main{StandardDirectoryReader(segments_2:3:nrt _0(4.1):C1)}
[junit4:junit4]   2> 9792 T96 oasc.CoreContainer.register replacing core: collection1
[junit4:junit4]   2> 9792 T96 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@3d88eb46
[junit4:junit4]   2> 9797 T96 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=1,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 9797 T96 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 9799 T81 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C15 name=collection1 org.apache.solr.core.SolrCore@3136505b
[junit4:junit4]   2> 9816 T94 C15 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 9827 T94 C15 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@680e6039 lockFactory=org.apache.lucene.store.NativeFSLockFactory@60e5272b; maxCacheMB=0.8359375 maxMergeSizeMB=0.57421875))),segFN=segments_2,generation=2,filenames=[_0.fnm, _0_TestBloomFilteredLucene41Postings_0.blm, _0_MockSep_0.pos, _0_Lucene41_0.pos, _0_TestBloomFilteredLucene41Postings_0.tim, _0_nrm.cfs, _0_TestBloomFilteredLucene41Postings_0.tip, _0_TestBloomFilteredLucene41Postings_0.pos, _0_MockSep_0.tib, _0_Lucene41_0.doc, _0_MockSep_0.skp, _0_nrm.cfe, _0_TestBloomFilteredLucene41Postings_0.doc, _0_MockSep_0.frq, segments_2, _0.fdx, _0_MockSep_0.tii, _0.si, _0_MockSep_0.pyl, _0_MockSep_0.doc, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@680e6039 lockFactory=org.apache.lucene.store.NativeFSLockFactory@60e5272b; maxCacheMB=0.8359375 maxMergeSizeMB=0.57421875))),segFN=segments_3,generation=3,filenames=[_1_MockSep_0.doc, _1_MockSep_0.frq, _1_MockSep_0.pyl, _1_nrm.cfs, _1_TestBloomFilteredLucene41Postings_0.pos, _1_Lucene41_0.doc, _1_MockSep_0.skp, _1_MockSep_0.pos, _1_TestBloomFilteredLucene41Postings_0.tim, _1_MockSep_0.tib, _1_Lucene41_0.tim, _1_Lucene41_0.tip, _1_TestBloomFilteredLucene41Postings_0.doc, _1_TestBloomFilteredLucene41Postings_0.tip, _1.fnm, _1_MockSep_0.tii, _1_nrm.cfe, _1.fdx, _1_Lucene41_0.pos, _1_TestBloomFilteredLucene41Postings_0.blm, _1.fdt, _1.si, segments_3]
[junit4:junit4]   2> 9827 T94 C15 oasc.SolrDeletionPolicy.updateCommits newest commit = 3
[junit4:junit4]   2> 9829 T94 C15 oass.SolrIndexSearcher.<init> Opening Searcher@1de9a4d0 main
[junit4:junit4]   2> 9830 T94 C15 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 9830 T94 C15 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 9832 T98 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1de9a4d0 main{StandardDirectoryReader(segments_3:7:nrt _1(4.1):C1)}
[junit4:junit4]   2> 9832 T94 C15 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {add=[42],commit=} 0 18
[junit4:junit4]   2> 9833 T81 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9843 T95 C15 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=teststop:terma&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 9844 T81 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9857 T96 C15 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=teststop:stopworda&version=2} hits=1 status=0 QTime=2 
[junit4:junit4]   2> 9858 T81 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9868 T94 C15 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=teststop:stopwordb&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 9869 T81 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9878 T95 C15 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=teststop:stopwordc&version=2} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 9924 T81 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1932761421
[junit4:junit4]   2> 9924 T81 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@3136505b
[junit4:junit4]   2> 9933 T81 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=1,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 9934 T81 oasc.SolrCore.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 9934 T81 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 9934 T81 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 9936 T81 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 9944 T81 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 10045 T81 oejut.QueuedThreadPool.doStop WARNING 4 threads could not be stopped
[junit4:junit4]   2> 10094 T81 oasu.AbstractSolrTestCase.tearDown ####TEARDOWN_START testStopwordsAfterCoreReload
[junit4:junit4]   2> 10095 T81 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=75451581
[junit4:junit4]   2> 10095 T81 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@41870e99
[junit4:junit4]   2> 10097 T81 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4]   2> 10097 T81 oasc.SolrCore.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 10098 T81 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 10098 T81 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 10098 T81 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 10112 T80 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 4 leaked thread(s).
[junit4:junit4]   2> 30240 T80 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 4 threads leaked from SUITE scope at org.apache.solr.AnalysisAfterCoreReloadTest: 
[junit4:junit4]   2> 	   1) Thread[id=92, name=qtp1156597185-92 Acceptor3 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   2) Thread[id=91, name=qtp1156597185-91 Acceptor2 SelectChannelConnector@0.0.0.0:29439, state=RUNNABLE, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   3) Thread[id=89, name=qtp1156597185-89 Acceptor0 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   4) Thread[id=90, name=qtp1156597185-90 Acceptor1 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 30241 T80 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
[junit4:junit4]   2> 	   1) Thread[id=92, name=qtp1156597185-92 Acceptor3 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]   2> 	   2) Thread[id=91, name=qtp1156597185-91 Acceptor2 SelectChannelConnector@0.0.0.0:29439, state=RUNNABLE, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]   2> 	   3) Thread[id=89, name=qtp1156597185-89 Acceptor0 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]   2> 	   4) Thread[id=90, name=qtp1156597185-90 Acceptor1 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]   2> 33284 T80 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
[junit4:junit4]   2> 	   1) Thread[id=92, name=qtp1156597185-92 Acceptor3 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   2) Thread[id=91, name=qtp1156597185-91 Acceptor2 SelectChannelConnector@0.0.0.0:29439, state=RUNNABLE, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   3) Thread[id=89, name=qtp1156597185-89 Acceptor0 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   4) Thread[id=90, name=qtp1156597185-90 Acceptor1 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene41: {timestamp=Lucene41(blocksize=128), range_facet_l=Lucene41(blocksize=128), id=PostingsFormat(name=MockSep), range_facet_sl=PostingsFormat(name=MockSep), range_facet_si=Lucene41(blocksize=128), teststop=Lucene41(blocksize=128), multiDefault=Lucene41(blocksize=128), intDefault=PostingsFormat(name=TestBloomFilteredLucene41Postings)}, sim=DefaultSimilarity, locale=ar_BH, timezone=CAT
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=5,free=77474648,total=130023424
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestLuceneMatchVersion, UniqFieldsUpdateProcessorFactoryTest, AutoCommitTest, MoreLikeThisHandlerTest, TestCoreContainer, FastVectorHighlighterTest, TestGroupingSearch, QueryElevationComponentTest, AnalysisAfterCoreReloadTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=AnalysisAfterCoreReloadTest -Dtests.seed=EFEC412AFD1FA1E8 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar_BH -Dtests.timezone=CAT -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR   0.00s J0 | AnalysisAfterCoreReloadTest (suite) <<<
[junit4:junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.AnalysisAfterCoreReloadTest: 
[junit4:junit4]    >    1) Thread[id=92, name=qtp1156597185-92 Acceptor3 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=91, name=qtp1156597185-91 Acceptor2 SelectChannelConnector@0.0.0.0:29439, state=RUNNABLE, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=89, name=qtp1156597185-89 Acceptor0 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=90, name=qtp1156597185-90 Acceptor1 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([EFEC412AFD1FA1E8]:0)
[junit4:junit4]    > Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4]    >    1) Thread[id=92, name=qtp1156597185-92 Acceptor3 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=91, name=qtp1156597185-91 Acceptor2 SelectChannelConnector@0.0.0.0:29439, state=RUNNABLE, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=89, name=qtp1156597185-89 Acceptor0 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=90, name=qtp1156597185-90 Acceptor1 SelectChannelConnector@0.0.0.0:29439, state=BLOCKED, group=TGRP-AnalysisAfterCoreReloadTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([EFEC412AFD1FA1E8]:0)
[junit4:junit4] Completed on J0 in 28.40s, 1 test, 2 errors <<< FAILURES!

[...truncated 152 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:335: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/module-build.xml:61: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:1163: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:827: There were test failures: 233 suites, 970 tests, 4 suite-level errors, 826 ignored

Total time: 28 minutes 8 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 614 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/614/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.TestDistributedGrouping

Error Message:
36 threads leaked from SUITE scope at org.apache.solr.TestDistributedGrouping:     1) Thread[id=91, name=qtp1758179843-91 Acceptor3 SelectChannelConnector@0.0.0.0:43789, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=53, name=qtp702834017-53 Acceptor0 SelectChannelConnector@0.0.0.0:16366, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=88, name=qtp1758179843-88 Acceptor0 SelectChannelConnector@0.0.0.0:43789, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=89, name=qtp1758179843-89 Acceptor1 SelectChannelConnector@0.0.0.0:43789, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=145, name=qtp979199720-145 Acceptor3 SelectChannelConnector@0.0.0.0:23576, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=54, name=qtp702834017-54 Acceptor1 SelectChannelConnector@0.0.0.0:16366, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=76, name=qtp778771945-76 Acceptor2 SelectChannelConnector@0.0.0.0:54577, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=143, name=qtp979199720-143 Acceptor1 SelectChannelConnector@0.0.0.0:23576, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=169, name=qtp1282480504-169 Acceptor1 SelectChannelConnector@0.0.0.0:62606, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=102, name=qtp284490634-102 Acceptor0 SelectChannelConnector@0.0.0.0:35039, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=104, name=qtp284490634-104 Acceptor2 SelectChannelConnector@0.0.0.0:35039, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=142, name=qtp979199720-142 Acceptor0 SelectChannelConnector@0.0.0.0:23576, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=157, name=qtp1165625747-157 Acceptor3 SelectChannelConnector@0.0.0.0:41277, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=156, name=qtp1165625747-156 Acceptor2 SelectChannelConnector@0.0.0.0:41277, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=155, name=qtp1165625747-155 Acceptor1 SelectChannelConnector@0.0.0.0:41277, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=75, name=qtp778771945-75 Acceptor1 SelectChannelConnector@0.0.0.0:54577, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=90, name=qtp1758179843-90 Acceptor2 SelectChannelConnector@0.0.0.0:43789, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=171, name=qtp1282480504-171 Acceptor3 SelectChannelConnector@0.0.0.0:62606, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=154, name=qtp1165625747-154 Acceptor0 SelectChannelConnector@0.0.0.0:41277, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=144, name=qtp979199720-144 Acceptor2 SelectChannelConnector@0.0.0.0:23576, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   21) Thread[id=130, name=qtp1864401185-130 Acceptor2 SelectChannelConnector@0.0.0.0:22609, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   22) Thread[id=77, name=qtp778771945-77 Acceptor3 SelectChannelConnector@0.0.0.0:54577, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   23) Thread[id=129, name=qtp1864401185-129 Acceptor1 SelectChannelConnector@0.0.0.0:22609, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   24) Thread[id=105, name=qtp284490634-105 Acceptor3 SelectChannelConnector@0.0.0.0:35039, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   25) Thread[id=168, name=qtp1282480504-168 Acceptor0 SelectChannelConnector@0.0.0.0:62606, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   26) Thread[id=42, name=qtp382843504-42 Acceptor3 SelectChannelConnector@0.0.0.0:53909, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   27) Thread[id=55, name=qtp702834017-55 Acceptor2 SelectChannelConnector@0.0.0.0:16366, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   28) Thread[id=39, name=qtp382843504-39 Acceptor0 SelectChannelConnector@0.0.0.0:53909, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   29) Thread[id=74, name=qtp778771945-74 Acceptor0 SelectChannelConnector@0.0.0.0:54577, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   30) Thread[id=56, name=qtp702834017-56 Acceptor3 SelectChannelConnector@0.0.0.0:16366, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   31) Thread[id=170, name=qtp1282480504-170 Acceptor2 SelectChannelConnector@0.0.0.0:62606, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   32) Thread[id=128, name=qtp1864401185-128 Acceptor0 SelectChannelConnector@0.0.0.0:22609, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   33) Thread[id=131, name=qtp1864401185-131 Acceptor3 SelectChannelConnector@0.0.0.0:22609, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   34) Thread[id=40, name=qtp382843504-40 Acceptor1 SelectChannelConnector@0.0.0.0:53909, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   35) Thread[id=103, name=qtp284490634-103 Acceptor1 SelectChannelConnector@0.0.0.0:35039, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   36) Thread[id=41, name=qtp382843504-41 Acceptor2 SelectChannelConnector@0.0.0.0:53909, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 36 threads leaked from SUITE scope at org.apache.solr.TestDistributedGrouping: 
   1) Thread[id=91, name=qtp1758179843-91 Acceptor3 SelectChannelConnector@0.0.0.0:43789, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=53, name=qtp702834017-53 Acceptor0 SelectChannelConnector@0.0.0.0:16366, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=88, name=qtp1758179843-88 Acceptor0 SelectChannelConnector@0.0.0.0:43789, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=89, name=qtp1758179843-89 Acceptor1 SelectChannelConnector@0.0.0.0:43789, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=145, name=qtp979199720-145 Acceptor3 SelectChannelConnector@0.0.0.0:23576, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=54, name=qtp702834017-54 Acceptor1 SelectChannelConnector@0.0.0.0:16366, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=76, name=qtp778771945-76 Acceptor2 SelectChannelConnector@0.0.0.0:54577, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=143, name=qtp979199720-143 Acceptor1 SelectChannelConnector@0.0.0.0:23576, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=169, name=qtp1282480504-169 Acceptor1 SelectChannelConnector@0.0.0.0:62606, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=102, name=qtp284490634-102 Acceptor0 SelectChannelConnector@0.0.0.0:35039, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=104, name=qtp284490634-104 Acceptor2 SelectChannelConnector@0.0.0.0:35039, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=142, name=qtp979199720-142 Acceptor0 SelectChannelConnector@0.0.0.0:23576, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=157, name=qtp1165625747-157 Acceptor3 SelectChannelConnector@0.0.0.0:41277, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=156, name=qtp1165625747-156 Acceptor2 SelectChannelConnector@0.0.0.0:41277, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=155, name=qtp1165625747-155 Acceptor1 SelectChannelConnector@0.0.0.0:41277, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=75, name=qtp778771945-75 Acceptor1 SelectChannelConnector@0.0.0.0:54577, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=90, name=qtp1758179843-90 Acceptor2 SelectChannelConnector@0.0.0.0:43789, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=171, name=qtp1282480504-171 Acceptor3 SelectChannelConnector@0.0.0.0:62606, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=154, name=qtp1165625747-154 Acceptor0 SelectChannelConnector@0.0.0.0:41277, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=144, name=qtp979199720-144 Acceptor2 SelectChannelConnector@0.0.0.0:23576, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  21) Thread[id=130, name=qtp1864401185-130 Acceptor2 SelectChannelConnector@0.0.0.0:22609, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  22) Thread[id=77, name=qtp778771945-77 Acceptor3 SelectChannelConnector@0.0.0.0:54577, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  23) Thread[id=129, name=qtp1864401185-129 Acceptor1 SelectChannelConnector@0.0.0.0:22609, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  24) Thread[id=105, name=qtp284490634-105 Acceptor3 SelectChannelConnector@0.0.0.0:35039, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  25) Thread[id=168, name=qtp1282480504-168 Acceptor0 SelectChannelConnector@0.0.0.0:62606, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  26) Thread[id=42, name=qtp382843504-42 Acceptor3 SelectChannelConnector@0.0.0.0:53909, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  27) Thread[id=55, name=qtp702834017-55 Acceptor2 SelectChannelConnector@0.0.0.0:16366, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  28) Thread[id=39, name=qtp382843504-39 Acceptor0 SelectChannelConnector@0.0.0.0:53909, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  29) Thread[id=74, name=qtp778771945-74 Acceptor0 SelectChannelConnector@0.0.0.0:54577, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  30) Thread[id=56, name=qtp702834017-56 Acceptor3 SelectChannelConnector@0.0.0.0:16366, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  31) Thread[id=170, name=qtp1282480504-170 Acceptor2 SelectChannelConnector@0.0.0.0:62606, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  32) Thread[id=128, name=qtp1864401185-128 Acceptor0 SelectChannelConnector@0.0.0.0:22609, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  33) Thread[id=131, name=qtp1864401185-131 Acceptor3 SelectChannelConnector@0.0.0.0:22609, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  34) Thread[id=40, name=qtp382843504-40 Acceptor1 SelectChannelConnector@0.0.0.0:53909, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  35) Thread[id=103, name=qtp284490634-103 Acceptor1 SelectChannelConnector@0.0.0.0:35039, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  36) Thread[id=41, name=qtp382843504-41 Acceptor2 SelectChannelConnector@0.0.0.0:53909, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([E10CCE4D9DEDE79A]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.TestDistributedGrouping

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=91, name=qtp1758179843-91 Acceptor3 SelectChannelConnector@0.0.0.0:43789, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=53, name=qtp702834017-53 Acceptor0 SelectChannelConnector@0.0.0.0:16366, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=88, name=qtp1758179843-88 Acceptor0 SelectChannelConnector@0.0.0.0:43789, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=89, name=qtp1758179843-89 Acceptor1 SelectChannelConnector@0.0.0.0:43789, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=145, name=qtp979199720-145 Acceptor3 SelectChannelConnector@0.0.0.0:23576, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=54, name=qtp702834017-54 Acceptor1 SelectChannelConnector@0.0.0.0:16366, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=76, name=qtp778771945-76 Acceptor2 SelectChannelConnector@0.0.0.0:54577, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=143, name=qtp979199720-143 Acceptor1 SelectChannelConnector@0.0.0.0:23576, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=169, name=qtp1282480504-169 Acceptor1 SelectChannelConnector@0.0.0.0:62606, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=102, name=qtp284490634-102 Acceptor0 SelectChannelConnector@0.0.0.0:35039, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=104, name=qtp284490634-104 Acceptor2 SelectChannelConnector@0.0.0.0:35039, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=142, name=qtp979199720-142 Acceptor0 SelectChannelConnector@0.0.0.0:23576, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=157, name=qtp1165625747-157 Acceptor3 SelectChannelConnector@0.0.0.0:41277, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=155, name=qtp1165625747-155 Acceptor1 SelectChannelConnector@0.0.0.0:41277, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=156, name=qtp1165625747-156 Acceptor2 SelectChannelConnector@0.0.0.0:41277, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=75, name=qtp778771945-75 Acceptor1 SelectChannelConnector@0.0.0.0:54577, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=90, name=qtp1758179843-90 Acceptor2 SelectChannelConnector@0.0.0.0:43789, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=144, name=qtp979199720-144 Acceptor2 SelectChannelConnector@0.0.0.0:23576, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=154, name=qtp1165625747-154 Acceptor0 SelectChannelConnector@0.0.0.0:41277, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=171, name=qtp1282480504-171 Acceptor3 SelectChannelConnector@0.0.0.0:62606, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   21) Thread[id=77, name=qtp778771945-77 Acceptor3 SelectChannelConnector@0.0.0.0:54577, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   22) Thread[id=130, name=qtp1864401185-130 Acceptor2 SelectChannelConnector@0.0.0.0:22609, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   23) Thread[id=105, name=qtp284490634-105 Acceptor3 SelectChannelConnector@0.0.0.0:35039, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   24) Thread[id=129, name=qtp1864401185-129 Acceptor1 SelectChannelConnector@0.0.0.0:22609, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   25) Thread[id=168, name=qtp1282480504-168 Acceptor0 SelectChannelConnector@0.0.0.0:62606, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   26) Thread[id=42, name=qtp382843504-42 Acceptor3 SelectChannelConnector@0.0.0.0:53909, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   27) Thread[id=55, name=qtp702834017-55 Acceptor2 SelectChannelConnector@0.0.0.0:16366, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   28) Thread[id=39, name=qtp382843504-39 Acceptor0 SelectChannelConnector@0.0.0.0:53909, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   29) Thread[id=74, name=qtp778771945-74 Acceptor0 SelectChannelConnector@0.0.0.0:54577, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   30) Thread[id=56, name=qtp702834017-56 Acceptor3 SelectChannelConnector@0.0.0.0:16366, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   31) Thread[id=170, name=qtp1282480504-170 Acceptor2 SelectChannelConnector@0.0.0.0:62606, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   32) Thread[id=128, name=qtp1864401185-128 Acceptor0 SelectChannelConnector@0.0.0.0:22609, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   33) Thread[id=131, name=qtp1864401185-131 Acceptor3 SelectChannelConnector@0.0.0.0:22609, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   34) Thread[id=40, name=qtp382843504-40 Acceptor1 SelectChannelConnector@0.0.0.0:53909, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   35) Thread[id=103, name=qtp284490634-103 Acceptor1 SelectChannelConnector@0.0.0.0:35039, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   36) Thread[id=41, name=qtp382843504-41 Acceptor2 SelectChannelConnector@0.0.0.0:53909, state=BLOCKED, group=TGRP-TestDistributedGrouping]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=91, name=qtp1758179843-91 Acceptor3 SelectChannelConnector@0.0.0.0:43789, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=53, name=qtp702834017-53 Acceptor0 SelectChannelConnector@0.0.0.0:16366, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=88, name=qtp1758179843-88 Acceptor0 SelectChannelConnector@0.0.0.0:43789, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=89, name=qtp1758179843-89 Acceptor1 SelectChannelConnector@0.0.0.0:43789, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=145, name=qtp979199720-145 Acceptor3 SelectChannelConnector@0.0.0.0:23576, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=54, name=qtp702834017-54 Acceptor1 SelectChannelConnector@0.0.0.0:16366, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=76, name=qtp778771945-76 Acceptor2 SelectChannelConnector@0.0.0.0:54577, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=143, name=qtp979199720-143 Acceptor1 SelectChannelConnector@0.0.0.0:23576, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=169, name=qtp1282480504-169 Acceptor1 SelectChannelConnector@0.0.0.0:62606, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=102, name=qtp284490634-102 Acceptor0 SelectChannelConnector@0.0.0.0:35039, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=104, name=qtp284490634-104 Acceptor2 SelectChannelConnector@0.0.0.0:35039, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=142, name=qtp979199720-142 Acceptor0 SelectChannelConnector@0.0.0.0:23576, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=157, name=qtp1165625747-157 Acceptor3 SelectChannelConnector@0.0.0.0:41277, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=155, name=qtp1165625747-155 Acceptor1 SelectChannelConnector@0.0.0.0:41277, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=156, name=qtp1165625747-156 Acceptor2 SelectChannelConnector@0.0.0.0:41277, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=75, name=qtp778771945-75 Acceptor1 SelectChannelConnector@0.0.0.0:54577, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=90, name=qtp1758179843-90 Acceptor2 SelectChannelConnector@0.0.0.0:43789, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=144, name=qtp979199720-144 Acceptor2 SelectChannelConnector@0.0.0.0:23576, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=154, name=qtp1165625747-154 Acceptor0 SelectChannelConnector@0.0.0.0:41277, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=171, name=qtp1282480504-171 Acceptor3 SelectChannelConnector@0.0.0.0:62606, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  21) Thread[id=77, name=qtp778771945-77 Acceptor3 SelectChannelConnector@0.0.0.0:54577, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  22) Thread[id=130, name=qtp1864401185-130 Acceptor2 SelectChannelConnector@0.0.0.0:22609, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  23) Thread[id=105, name=qtp284490634-105 Acceptor3 SelectChannelConnector@0.0.0.0:35039, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  24) Thread[id=129, name=qtp1864401185-129 Acceptor1 SelectChannelConnector@0.0.0.0:22609, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  25) Thread[id=168, name=qtp1282480504-168 Acceptor0 SelectChannelConnector@0.0.0.0:62606, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  26) Thread[id=42, name=qtp382843504-42 Acceptor3 SelectChannelConnector@0.0.0.0:53909, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  27) Thread[id=55, name=qtp702834017-55 Acceptor2 SelectChannelConnector@0.0.0.0:16366, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  28) Thread[id=39, name=qtp382843504-39 Acceptor0 SelectChannelConnector@0.0.0.0:53909, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  29) Thread[id=74, name=qtp778771945-74 Acceptor0 SelectChannelConnector@0.0.0.0:54577, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  30) Thread[id=56, name=qtp702834017-56 Acceptor3 SelectChannelConnector@0.0.0.0:16366, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  31) Thread[id=170, name=qtp1282480504-170 Acceptor2 SelectChannelConnector@0.0.0.0:62606, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  32) Thread[id=128, name=qtp1864401185-128 Acceptor0 SelectChannelConnector@0.0.0.0:22609, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  33) Thread[id=131, name=qtp1864401185-131 Acceptor3 SelectChannelConnector@0.0.0.0:22609, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  34) Thread[id=40, name=qtp382843504-40 Acceptor1 SelectChannelConnector@0.0.0.0:53909, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  35) Thread[id=103, name=qtp284490634-103 Acceptor1 SelectChannelConnector@0.0.0.0:35039, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  36) Thread[id=41, name=qtp382843504-41 Acceptor2 SelectChannelConnector@0.0.0.0:53909, state=BLOCKED, group=TGRP-TestDistributedGrouping]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([E10CCE4D9DEDE79A]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.component.DistributedSpellCheckComponentTest

Error Message:
36 threads leaked from SUITE scope at org.apache.solr.handler.component.DistributedSpellCheckComponentTest:     1) Thread[id=373, name=qtp1166599758-373 Acceptor1 SelectChannelConnector@0.0.0.0:16534, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=328, name=qtp869068009-328 Acceptor0 SelectChannelConnector@0.0.0.0:36572, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=420, name=qtp2066373459-420 Acceptor1 SelectChannelConnector@0.0.0.0:32002, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=421, name=qtp2066373459-421 Acceptor2 SelectChannelConnector@0.0.0.0:32002, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=360, name=qtp1359724161-360 Acceptor2 SelectChannelConnector@0.0.0.0:10890, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=406, name=qtp279283093-406 Acceptor0 SelectChannelConnector@0.0.0.0:35826, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=449, name=qtp1011276511-449 Acceptor2 SelectChannelConnector@0.0.0.0:56217, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=342, name=qtp1562851016-342 Acceptor0 SelectChannelConnector@0.0.0.0:56921, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=344, name=qtp1562851016-344 Acceptor2 SelectChannelConnector@0.0.0.0:56921, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=388, name=qtp1396210494-388 Acceptor2 SelectChannelConnector@0.0.0.0:64726, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=447, name=qtp1011276511-447 Acceptor0 SelectChannelConnector@0.0.0.0:56217, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=345, name=qtp1562851016-345 Acceptor3 SelectChannelConnector@0.0.0.0:56921, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=434, name=qtp1951812991-434 Acceptor1 SelectChannelConnector@0.0.0.0:46423, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=372, name=qtp1166599758-372 Acceptor0 SelectChannelConnector@0.0.0.0:16534, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=433, name=qtp1951812991-433 Acceptor0 SelectChannelConnector@0.0.0.0:46423, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=330, name=qtp869068009-330 Acceptor2 SelectChannelConnector@0.0.0.0:36572, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=359, name=qtp1359724161-359 Acceptor1 SelectChannelConnector@0.0.0.0:10890, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=358, name=qtp1359724161-358 Acceptor0 SelectChannelConnector@0.0.0.0:10890, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=361, name=qtp1359724161-361 Acceptor3 SelectChannelConnector@0.0.0.0:10890, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=422, name=qtp2066373459-422 Acceptor3 SelectChannelConnector@0.0.0.0:32002, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   21) Thread[id=343, name=qtp1562851016-343 Acceptor1 SelectChannelConnector@0.0.0.0:56921, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   22) Thread[id=450, name=qtp1011276511-450 Acceptor3 SelectChannelConnector@0.0.0.0:56217, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   23) Thread[id=419, name=qtp2066373459-419 Acceptor0 SelectChannelConnector@0.0.0.0:32002, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   24) Thread[id=448, name=qtp1011276511-448 Acceptor1 SelectChannelConnector@0.0.0.0:56217, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   25) Thread[id=329, name=qtp869068009-329 Acceptor1 SelectChannelConnector@0.0.0.0:36572, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   26) Thread[id=331, name=qtp869068009-331 Acceptor3 SelectChannelConnector@0.0.0.0:36572, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   27) Thread[id=389, name=qtp1396210494-389 Acceptor3 SelectChannelConnector@0.0.0.0:64726, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   28) Thread[id=386, name=qtp1396210494-386 Acceptor0 SelectChannelConnector@0.0.0.0:64726, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   29) Thread[id=407, name=qtp279283093-407 Acceptor1 SelectChannelConnector@0.0.0.0:35826, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   30) Thread[id=409, name=qtp279283093-409 Acceptor3 SelectChannelConnector@0.0.0.0:35826, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   31) Thread[id=375, name=qtp1166599758-375 Acceptor3 SelectChannelConnector@0.0.0.0:16534, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   32) Thread[id=408, name=qtp279283093-408 Acceptor2 SelectChannelConnector@0.0.0.0:35826, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   33) Thread[id=436, name=qtp1951812991-436 Acceptor3 SelectChannelConnector@0.0.0.0:46423, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   34) Thread[id=374, name=qtp1166599758-374 Acceptor2 SelectChannelConnector@0.0.0.0:16534, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   35) Thread[id=387, name=qtp1396210494-387 Acceptor1 SelectChannelConnector@0.0.0.0:64726, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   36) Thread[id=435, name=qtp1951812991-435 Acceptor2 SelectChannelConnector@0.0.0.0:46423, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 36 threads leaked from SUITE scope at org.apache.solr.handler.component.DistributedSpellCheckComponentTest: 
   1) Thread[id=373, name=qtp1166599758-373 Acceptor1 SelectChannelConnector@0.0.0.0:16534, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=328, name=qtp869068009-328 Acceptor0 SelectChannelConnector@0.0.0.0:36572, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=420, name=qtp2066373459-420 Acceptor1 SelectChannelConnector@0.0.0.0:32002, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=421, name=qtp2066373459-421 Acceptor2 SelectChannelConnector@0.0.0.0:32002, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=360, name=qtp1359724161-360 Acceptor2 SelectChannelConnector@0.0.0.0:10890, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=406, name=qtp279283093-406 Acceptor0 SelectChannelConnector@0.0.0.0:35826, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=449, name=qtp1011276511-449 Acceptor2 SelectChannelConnector@0.0.0.0:56217, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=342, name=qtp1562851016-342 Acceptor0 SelectChannelConnector@0.0.0.0:56921, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=344, name=qtp1562851016-344 Acceptor2 SelectChannelConnector@0.0.0.0:56921, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=388, name=qtp1396210494-388 Acceptor2 SelectChannelConnector@0.0.0.0:64726, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=447, name=qtp1011276511-447 Acceptor0 SelectChannelConnector@0.0.0.0:56217, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=345, name=qtp1562851016-345 Acceptor3 SelectChannelConnector@0.0.0.0:56921, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=434, name=qtp1951812991-434 Acceptor1 SelectChannelConnector@0.0.0.0:46423, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=372, name=qtp1166599758-372 Acceptor0 SelectChannelConnector@0.0.0.0:16534, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=433, name=qtp1951812991-433 Acceptor0 SelectChannelConnector@0.0.0.0:46423, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=330, name=qtp869068009-330 Acceptor2 SelectChannelConnector@0.0.0.0:36572, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=359, name=qtp1359724161-359 Acceptor1 SelectChannelConnector@0.0.0.0:10890, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=358, name=qtp1359724161-358 Acceptor0 SelectChannelConnector@0.0.0.0:10890, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=361, name=qtp1359724161-361 Acceptor3 SelectChannelConnector@0.0.0.0:10890, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=422, name=qtp2066373459-422 Acceptor3 SelectChannelConnector@0.0.0.0:32002, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  21) Thread[id=343, name=qtp1562851016-343 Acceptor1 SelectChannelConnector@0.0.0.0:56921, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  22) Thread[id=450, name=qtp1011276511-450 Acceptor3 SelectChannelConnector@0.0.0.0:56217, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  23) Thread[id=419, name=qtp2066373459-419 Acceptor0 SelectChannelConnector@0.0.0.0:32002, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  24) Thread[id=448, name=qtp1011276511-448 Acceptor1 SelectChannelConnector@0.0.0.0:56217, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  25) Thread[id=329, name=qtp869068009-329 Acceptor1 SelectChannelConnector@0.0.0.0:36572, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  26) Thread[id=331, name=qtp869068009-331 Acceptor3 SelectChannelConnector@0.0.0.0:36572, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  27) Thread[id=389, name=qtp1396210494-389 Acceptor3 SelectChannelConnector@0.0.0.0:64726, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  28) Thread[id=386, name=qtp1396210494-386 Acceptor0 SelectChannelConnector@0.0.0.0:64726, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  29) Thread[id=407, name=qtp279283093-407 Acceptor1 SelectChannelConnector@0.0.0.0:35826, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  30) Thread[id=409, name=qtp279283093-409 Acceptor3 SelectChannelConnector@0.0.0.0:35826, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  31) Thread[id=375, name=qtp1166599758-375 Acceptor3 SelectChannelConnector@0.0.0.0:16534, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  32) Thread[id=408, name=qtp279283093-408 Acceptor2 SelectChannelConnector@0.0.0.0:35826, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  33) Thread[id=436, name=qtp1951812991-436 Acceptor3 SelectChannelConnector@0.0.0.0:46423, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  34) Thread[id=374, name=qtp1166599758-374 Acceptor2 SelectChannelConnector@0.0.0.0:16534, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  35) Thread[id=387, name=qtp1396210494-387 Acceptor1 SelectChannelConnector@0.0.0.0:64726, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  36) Thread[id=435, name=qtp1951812991-435 Acceptor2 SelectChannelConnector@0.0.0.0:46423, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([E10CCE4D9DEDE79A]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.component.DistributedSpellCheckComponentTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=373, name=qtp1166599758-373 Acceptor1 SelectChannelConnector@0.0.0.0:16534, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=328, name=qtp869068009-328 Acceptor0 SelectChannelConnector@0.0.0.0:36572, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=420, name=qtp2066373459-420 Acceptor1 SelectChannelConnector@0.0.0.0:32002, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=421, name=qtp2066373459-421 Acceptor2 SelectChannelConnector@0.0.0.0:32002, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=360, name=qtp1359724161-360 Acceptor2 SelectChannelConnector@0.0.0.0:10890, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=406, name=qtp279283093-406 Acceptor0 SelectChannelConnector@0.0.0.0:35826, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=449, name=qtp1011276511-449 Acceptor2 SelectChannelConnector@0.0.0.0:56217, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=342, name=qtp1562851016-342 Acceptor0 SelectChannelConnector@0.0.0.0:56921, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=344, name=qtp1562851016-344 Acceptor2 SelectChannelConnector@0.0.0.0:56921, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=388, name=qtp1396210494-388 Acceptor2 SelectChannelConnector@0.0.0.0:64726, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=447, name=qtp1011276511-447 Acceptor0 SelectChannelConnector@0.0.0.0:56217, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=345, name=qtp1562851016-345 Acceptor3 SelectChannelConnector@0.0.0.0:56921, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=372, name=qtp1166599758-372 Acceptor0 SelectChannelConnector@0.0.0.0:16534, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=434, name=qtp1951812991-434 Acceptor1 SelectChannelConnector@0.0.0.0:46423, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=433, name=qtp1951812991-433 Acceptor0 SelectChannelConnector@0.0.0.0:46423, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=330, name=qtp869068009-330 Acceptor2 SelectChannelConnector@0.0.0.0:36572, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=359, name=qtp1359724161-359 Acceptor1 SelectChannelConnector@0.0.0.0:10890, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=358, name=qtp1359724161-358 Acceptor0 SelectChannelConnector@0.0.0.0:10890, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=361, name=qtp1359724161-361 Acceptor3 SelectChannelConnector@0.0.0.0:10890, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=422, name=qtp2066373459-422 Acceptor3 SelectChannelConnector@0.0.0.0:32002, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   21) Thread[id=343, name=qtp1562851016-343 Acceptor1 SelectChannelConnector@0.0.0.0:56921, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   22) Thread[id=419, name=qtp2066373459-419 Acceptor0 SelectChannelConnector@0.0.0.0:32002, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   23) Thread[id=450, name=qtp1011276511-450 Acceptor3 SelectChannelConnector@0.0.0.0:56217, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   24) Thread[id=448, name=qtp1011276511-448 Acceptor1 SelectChannelConnector@0.0.0.0:56217, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   25) Thread[id=329, name=qtp869068009-329 Acceptor1 SelectChannelConnector@0.0.0.0:36572, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   26) Thread[id=331, name=qtp869068009-331 Acceptor3 SelectChannelConnector@0.0.0.0:36572, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   27) Thread[id=389, name=qtp1396210494-389 Acceptor3 SelectChannelConnector@0.0.0.0:64726, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   28) Thread[id=386, name=qtp1396210494-386 Acceptor0 SelectChannelConnector@0.0.0.0:64726, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   29) Thread[id=407, name=qtp279283093-407 Acceptor1 SelectChannelConnector@0.0.0.0:35826, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   30) Thread[id=409, name=qtp279283093-409 Acceptor3 SelectChannelConnector@0.0.0.0:35826, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   31) Thread[id=375, name=qtp1166599758-375 Acceptor3 SelectChannelConnector@0.0.0.0:16534, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   32) Thread[id=408, name=qtp279283093-408 Acceptor2 SelectChannelConnector@0.0.0.0:35826, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   33) Thread[id=436, name=qtp1951812991-436 Acceptor3 SelectChannelConnector@0.0.0.0:46423, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   34) Thread[id=374, name=qtp1166599758-374 Acceptor2 SelectChannelConnector@0.0.0.0:16534, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   35) Thread[id=387, name=qtp1396210494-387 Acceptor1 SelectChannelConnector@0.0.0.0:64726, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   36) Thread[id=435, name=qtp1951812991-435 Acceptor2 SelectChannelConnector@0.0.0.0:46423, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=373, name=qtp1166599758-373 Acceptor1 SelectChannelConnector@0.0.0.0:16534, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=328, name=qtp869068009-328 Acceptor0 SelectChannelConnector@0.0.0.0:36572, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=420, name=qtp2066373459-420 Acceptor1 SelectChannelConnector@0.0.0.0:32002, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=421, name=qtp2066373459-421 Acceptor2 SelectChannelConnector@0.0.0.0:32002, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=360, name=qtp1359724161-360 Acceptor2 SelectChannelConnector@0.0.0.0:10890, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=406, name=qtp279283093-406 Acceptor0 SelectChannelConnector@0.0.0.0:35826, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=449, name=qtp1011276511-449 Acceptor2 SelectChannelConnector@0.0.0.0:56217, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=342, name=qtp1562851016-342 Acceptor0 SelectChannelConnector@0.0.0.0:56921, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=344, name=qtp1562851016-344 Acceptor2 SelectChannelConnector@0.0.0.0:56921, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=388, name=qtp1396210494-388 Acceptor2 SelectChannelConnector@0.0.0.0:64726, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=447, name=qtp1011276511-447 Acceptor0 SelectChannelConnector@0.0.0.0:56217, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=345, name=qtp1562851016-345 Acceptor3 SelectChannelConnector@0.0.0.0:56921, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=372, name=qtp1166599758-372 Acceptor0 SelectChannelConnector@0.0.0.0:16534, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=434, name=qtp1951812991-434 Acceptor1 SelectChannelConnector@0.0.0.0:46423, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=433, name=qtp1951812991-433 Acceptor0 SelectChannelConnector@0.0.0.0:46423, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=330, name=qtp869068009-330 Acceptor2 SelectChannelConnector@0.0.0.0:36572, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=359, name=qtp1359724161-359 Acceptor1 SelectChannelConnector@0.0.0.0:10890, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=358, name=qtp1359724161-358 Acceptor0 SelectChannelConnector@0.0.0.0:10890, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=361, name=qtp1359724161-361 Acceptor3 SelectChannelConnector@0.0.0.0:10890, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=422, name=qtp2066373459-422 Acceptor3 SelectChannelConnector@0.0.0.0:32002, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  21) Thread[id=343, name=qtp1562851016-343 Acceptor1 SelectChannelConnector@0.0.0.0:56921, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  22) Thread[id=419, name=qtp2066373459-419 Acceptor0 SelectChannelConnector@0.0.0.0:32002, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  23) Thread[id=450, name=qtp1011276511-450 Acceptor3 SelectChannelConnector@0.0.0.0:56217, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  24) Thread[id=448, name=qtp1011276511-448 Acceptor1 SelectChannelConnector@0.0.0.0:56217, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  25) Thread[id=329, name=qtp869068009-329 Acceptor1 SelectChannelConnector@0.0.0.0:36572, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native M

[...truncated too long message...]

ector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   27) Thread[id=389, name=qtp1396210494-389 Acceptor3 SelectChannelConnector@0.0.0.0:64726, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   28) Thread[id=386, name=qtp1396210494-386 Acceptor0 SelectChannelConnector@0.0.0.0:64726, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   29) Thread[id=407, name=qtp279283093-407 Acceptor1 SelectChannelConnector@0.0.0.0:35826, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   30) Thread[id=409, name=qtp279283093-409 Acceptor3 SelectChannelConnector@0.0.0.0:35826, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   31) Thread[id=375, name=qtp1166599758-375 Acceptor3 SelectChannelConnector@0.0.0.0:16534, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   32) Thread[id=408, name=qtp279283093-408 Acceptor2 SelectChannelConnector@0.0.0.0:35826, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   33) Thread[id=436, name=qtp1951812991-436 Acceptor3 SelectChannelConnector@0.0.0.0:46423, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   34) Thread[id=374, name=qtp1166599758-374 Acceptor2 SelectChannelConnector@0.0.0.0:16534, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   35) Thread[id=387, name=qtp1396210494-387 Acceptor1 SelectChannelConnector@0.0.0.0:64726, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   36) Thread[id=435, name=qtp1951812991-435 Acceptor2 SelectChannelConnector@0.0.0.0:46423, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([E10CCE4D9DEDE79A]:0)
[junit4:junit4] Completed on J0 in 38.90s, 1 test, 2 errors <<< FAILURES!

[...truncated 116 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:335: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/module-build.xml:61: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:1163: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:827: There were test failures: 233 suites, 970 tests, 4 suite-level errors, 807 ignored

Total time: 20 minutes 47 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 613 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/613/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
12 threads leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:     1) Thread[id=185, name=qtp507512929-185 Acceptor2 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=183, name=qtp507512929-183 Acceptor0 SelectChannelConnector@0.0.0.0:24328, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=165, name=qtp15205627-165 Acceptor1 SelectChannelConnector@0.0.0.0:18910, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=115, name=qtp1065037894-115 Acceptor0 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=184, name=qtp507512929-184 Acceptor1 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=118, name=qtp1065037894-118 Acceptor3 SelectChannelConnector@0.0.0.0:38672, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=166, name=qtp15205627-166 Acceptor2 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=117, name=qtp1065037894-117 Acceptor2 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=186, name=qtp507512929-186 Acceptor3 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=167, name=qtp15205627-167 Acceptor3 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=116, name=qtp1065037894-116 Acceptor1 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=164, name=qtp15205627-164 Acceptor0 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 12 threads leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest: 
   1) Thread[id=185, name=qtp507512929-185 Acceptor2 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=183, name=qtp507512929-183 Acceptor0 SelectChannelConnector@0.0.0.0:24328, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=165, name=qtp15205627-165 Acceptor1 SelectChannelConnector@0.0.0.0:18910, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=115, name=qtp1065037894-115 Acceptor0 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=184, name=qtp507512929-184 Acceptor1 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=118, name=qtp1065037894-118 Acceptor3 SelectChannelConnector@0.0.0.0:38672, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=166, name=qtp15205627-166 Acceptor2 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=117, name=qtp1065037894-117 Acceptor2 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=186, name=qtp507512929-186 Acceptor3 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=167, name=qtp15205627-167 Acceptor3 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=116, name=qtp1065037894-116 Acceptor1 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=164, name=qtp15205627-164 Acceptor0 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([C6F54FC7A3B9989F]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=185, name=qtp507512929-185 Acceptor2 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=183, name=qtp507512929-183 Acceptor0 SelectChannelConnector@0.0.0.0:24328, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=117, name=qtp1065037894-117 Acceptor2 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=165, name=qtp15205627-165 Acceptor1 SelectChannelConnector@0.0.0.0:18910, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=186, name=qtp507512929-186 Acceptor3 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=167, name=qtp15205627-167 Acceptor3 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=115, name=qtp1065037894-115 Acceptor0 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=184, name=qtp507512929-184 Acceptor1 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=116, name=qtp1065037894-116 Acceptor1 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=164, name=qtp15205627-164 Acceptor0 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=118, name=qtp1065037894-118 Acceptor3 SelectChannelConnector@0.0.0.0:38672, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=166, name=qtp15205627-166 Acceptor2 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=185, name=qtp507512929-185 Acceptor2 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=183, name=qtp507512929-183 Acceptor0 SelectChannelConnector@0.0.0.0:24328, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=117, name=qtp1065037894-117 Acceptor2 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=165, name=qtp15205627-165 Acceptor1 SelectChannelConnector@0.0.0.0:18910, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=186, name=qtp507512929-186 Acceptor3 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=167, name=qtp15205627-167 Acceptor3 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=115, name=qtp1065037894-115 Acceptor0 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=184, name=qtp507512929-184 Acceptor1 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=116, name=qtp1065037894-116 Acceptor1 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=164, name=qtp15205627-164 Acceptor0 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=118, name=qtp1065037894-118 Acceptor3 SelectChannelConnector@0.0.0.0:38672, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=166, name=qtp15205627-166 Acceptor2 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([C6F54FC7A3B9989F]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.request.TestRemoteStreaming

Error Message:
4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming:     1) Thread[id=329, name=qtp651274847-329 Acceptor1 SelectChannelConnector@0.0.0.0:६३०६६, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=331, name=qtp651274847-331 Acceptor3 SelectChannelConnector@0.0.0.0:६३०६६, state=RUNNABLE, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=330, name=qtp651274847-330 Acceptor2 SelectChannelConnector@0.0.0.0:६३०६६, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=328, name=qtp651274847-328 Acceptor0 SelectChannelConnector@0.0.0.0:६३०६६, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming: 
   1) Thread[id=329, name=qtp651274847-329 Acceptor1 SelectChannelConnector@0.0.0.0:६३०६६, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=331, name=qtp651274847-331 Acceptor3 SelectChannelConnector@0.0.0.0:६३०६६, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=330, name=qtp651274847-330 Acceptor2 SelectChannelConnector@0.0.0.0:६३०६६, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=328, name=qtp651274847-328 Acceptor0 SelectChannelConnector@0.0.0.0:६३०६६, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([C6F54FC7A3B9989F]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.request.TestRemoteStreaming

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=329, name=qtp651274847-329 Acceptor1 SelectChannelConnector@0.0.0.0:६३०६६, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=331, name=qtp651274847-331 Acceptor3 SelectChannelConnector@0.0.0.0:६३०६६, state=RUNNABLE, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=330, name=qtp651274847-330 Acceptor2 SelectChannelConnector@0.0.0.0:६३०६६, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=328, name=qtp651274847-328 Acceptor0 SelectChannelConnector@0.0.0.0:६३०६६, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=329, name=qtp651274847-329 Acceptor1 SelectChannelConnector@0.0.0.0:६३०६६, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=331, name=qtp651274847-331 Acceptor3 SelectChannelConnector@0.0.0.0:६३०६६, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=330, name=qtp651274847-330 Acceptor2 SelectChannelConnector@0.0.0.0:६३०६६, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=328, name=qtp651274847-328 Acceptor0 SelectChannelConnector@0.0.0.0:६३०६६, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([C6F54FC7A3B9989F]:0)




Build Log:
[...truncated 8416 lines...]
[junit4:junit4] Suite: org.apache.solr.request.TestRemoteStreaming
[junit4:junit4]   2> 0 T321 oas.SolrTestCaseJ4.initCore ####initCore
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-TestRemoteStreaming-1352218923210
[junit4:junit4]   2> 1 T321 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4:junit4]   2> 571 T321 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 951 T321 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:?????
[junit4:junit4]   2> 997 T321 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 997 T321 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: solr/
[junit4:junit4]   2> 998 T321 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: 'solr/'
[junit4:junit4]   2> 1045 T321 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1045 T321 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1046 T321 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: solr/
[junit4:junit4]   2> 1046 T321 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solr/solr.xml
[junit4:junit4]   2> 1046 T321 oasc.CoreContainer.<init> New CoreContainer 601419896
[junit4:junit4]   2> 1047 T321 oasc.CoreContainer$Initializer.initialize no solr.xml file found - using default
[junit4:junit4]   2> 1047 T321 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'solr/'
[junit4:junit4]   2> 1048 T321 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'solr/'
[junit4:junit4]   2> 1096 T321 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 1123 T321 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: solr/collection1
[junit4:junit4]   2> 1125 T321 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'solr/collection1/'
[junit4:junit4]   2> 1211 T321 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 1285 T321 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1291 T321 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 1302 T321 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1930 T321 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1942 T321 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1947 T321 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1978 T321 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1987 T321 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1992 T321 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1997 T321 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1999 T321 oasc.SolrCore.<init> [collection1] Opening new SolrCore at solr/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-TestRemoteStreaming-1352218923210/
[junit4:junit4]   2> 1999 T321 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@732d14e0
[junit4:junit4]   2> 2001 T321 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-TestRemoteStreaming-1352218923210/index/
[junit4:junit4]   2> 2001 T321 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-TestRemoteStreaming-1352218923210/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2003 T321 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-TestRemoteStreaming-1352218923210/index forceNew:false
[junit4:junit4]   2> 2005 T321 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@257938cb lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c9c5420),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 2005 T321 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 2006 T321 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 2009 T321 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2009 T321 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 2010 T321 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2010 T321 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 2011 T321 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2011 T321 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2012 T321 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2012 T321 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2013 T321 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2014 T321 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 2014 T321 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 2015 T321 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2015 T321 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 2016 T321 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 2016 T321 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 2017 T321 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2017 T321 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2018 T321 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2018 T321 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2019 T321 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2020 T321 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2020 T321 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2021 T321 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2021 T321 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 2022 T321 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 2034 T321 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2039 T321 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 2041 T321 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: []
[junit4:junit4]   2> 2044 T321 oass.SolrIndexSearcher.<init> Opening Searcher@11c9a329 main
[junit4:junit4]   2> 2044 T321 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 2045 T321 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2045 T321 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2046 T321 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 2079 T321 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 2123 T321 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 2124 T321 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 2124 T321 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 2124 T321 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 2124 T321 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 2125 T321 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 2125 T321 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 2125 T321 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 2125 T321 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 2126 T321 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 2141 T336 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@11c9a329 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2144 T321 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 2145 T321 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 2145 T321 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 2145 T321 oas.SolrJettyTestBase.createJetty Jetty Assigned Port#63066
[junit4:junit4]   2> 2151 T321 oas.SolrTestCaseJ4.setUp ###Starting testQtUpdateFails
[junit4:junit4]   2> 2209 T321 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C29 name=collection1 org.apache.solr.core.SolrCore@3e5d2347
[junit4:junit4]   2> 3090 T334 C29 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@257938cb lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c9c5420),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 3091 T334 C29 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 3098 T334 C29 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1234]} 0 17
[junit4:junit4]   2> 3228 T335 C29 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 3232 T335 C29 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@257938cb lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c9c5420),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@257938cb lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c9c5420),segFN=segments_2,generation=2,filenames=[_0_4.len, _0_1.len, _0_3.len, _0_5.len, segments_2, _0.inf, _0.si, _0.pst, _0.fld]
[junit4:junit4]   2> 3232 T335 C29 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 3237 T335 C29 oass.SolrIndexSearcher.<init> Opening Searcher@704c8f74 main
[junit4:junit4]   2> 3238 T335 C29 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3238 T335 C29 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 3240 T336 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@704c8f74 main{StandardDirectoryReader(segments_2:3 _0(4.1):C1)}
[junit4:junit4]   2> 3240 T335 C29 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 12
[junit4:junit4]   2> 3300 T333 C29 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=36 
[junit4:junit4]   2> 3308 T321 oas.SolrTestCaseJ4.tearDown ###Ending testQtUpdateFails
[junit4:junit4]   2> 3314 T321 oas.SolrTestCaseJ4.setUp ###Starting testMakeDeleteAllUrl
[junit4:junit4]   2> 3319 T335 C29 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1234]} 0 2
[junit4:junit4]   2> 3322 T333 C29 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 3331 T333 C29 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@257938cb lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c9c5420),segFN=segments_2,generation=2,filenames=[_0_4.len, _0_1.len, _0_3.len, _0_5.len, segments_2, _0.inf, _0.si, _0.pst, _0.fld]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@257938cb lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c9c5420),segFN=segments_3,generation=3,filenames=[_1_1.len, _1.pst, _1.fld, _1_3.len, _1.si, _1.inf, _1_4.len, _1_5.len, segments_3]
[junit4:junit4]   2> 3331 T333 C29 oasc.SolrDeletionPolicy.updateCommits newest commit = 3
[junit4:junit4]   2> 3333 T333 C29 oass.SolrIndexSearcher.<init> Opening Searcher@39a24033 main
[junit4:junit4]   2> 3333 T333 C29 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3333 T333 C29 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 3335 T336 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@39a24033 main{StandardDirectoryReader(segments_3:7 _1(4.1):C1)}
[junit4:junit4]   2> 3335 T333 C29 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 14
[junit4:junit4]   2> 3339 T334 C29 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 3347 T333 C29 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4:junit4]   2> 3348 T333 C29 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 3348 T333 C29 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@257938cb lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c9c5420),segFN=segments_3,generation=3,filenames=[_1_1.len, _1.pst, _1.fld, _1_3.len, _1.si, _1.inf, _1_4.len, _1_5.len, segments_3]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@257938cb lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c9c5420),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 3348 T333 C29 oasc.SolrDeletionPolicy.updateCommits newest commit = 4
[junit4:junit4]   2> 3349 T333 C29 oass.SolrIndexSearcher.<init> Opening Searcher@541eae08 main
[junit4:junit4]   2> 3349 T333 C29 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3350 T333 C29 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 3351 T336 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@541eae08 main{StandardDirectoryReader(segments_4:8)}
[junit4:junit4]   2> 3351 T333 C29 UPDATE [collection1] webapp=/solr path=/update params={commit=true&stream.body=<delete><query>*:*</query></delete>} {deleteByQuery=*:*,commit=} 0 4
[junit4:junit4]   2> 3365 T334 C29 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 3366 T321 oas.SolrTestCaseJ4.tearDown ###Ending testMakeDeleteAllUrl
[junit4:junit4]   2> 3372 T321 oas.SolrTestCaseJ4.setUp ###Starting testNoUrlAccess
[junit4:junit4]   2> 3377 T333 C29 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1234]} 0 1
[junit4:junit4]   2> 3381 T335 C29 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 3383 T335 C29 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@257938cb lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c9c5420),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@257938cb lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c9c5420),segFN=segments_5,generation=5,filenames=[_2.si, _2.pst, _2.fld, _2_5.len, _2.inf, _2_3.len, segments_5, _2_1.len, _2_4.len]
[junit4:junit4]   2> 3384 T335 C29 oasc.SolrDeletionPolicy.updateCommits newest commit = 5
[junit4:junit4]   2> 3436 T335 C29 oass.SolrIndexSearcher.<init> Opening Searcher@e71d24c main
[junit4:junit4]   2> 3436 T335 C29 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3437 T335 C29 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 3439 T336 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@e71d24c main{StandardDirectoryReader(segments_5:10 _2(4.1):C1)}
[junit4:junit4]   2> 3439 T335 C29 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 58
[junit4:junit4]   2> 3443 T333 C29 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 3448 T335 C29 REQ [collection1] webapp=/solr path=/select params={wt=javabin&stream.url=http://127.0.0.1:63066/solr/update?commit%3Dtrue%26stream.body%3D%253Cdelete%253E%253Cquery%253E*%253A*%253C%252Fquery%253E%253C%252Fdelete%253E&q=*:*&version=2} hits=1 status=0 QTime=0 
[junit4:junit4]   2> 3451 T334 C29 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=0 
[junit4:junit4]   2> 3452 T321 oas.SolrTestCaseJ4.tearDown ###Ending testNoUrlAccess
[junit4:junit4]   2> 3458 T321 oas.SolrTestCaseJ4.setUp ###Starting testStreamUrl
[junit4:junit4]   2> 3461 T333 C29 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1234]} 0 1
[junit4:junit4]   2> 3465 T335 C29 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 3519 T335 C29 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@257938cb lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c9c5420),segFN=segments_5,generation=5,filenames=[_2.si, _2.pst, _2.fld, _2_5.len, _2.inf, _2_3.len, segments_5, _2_1.len, _2_4.len]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@257938cb lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c9c5420),segFN=segments_6,generation=6,filenames=[_3.fld, _3.inf, _3_5.len, _3_4.len, _3.si, _3_3.len, segments_6, _3.pst, _3_1.len]
[junit4:junit4]   2> 3519 T335 C29 oasc.SolrDeletionPolicy.updateCommits newest commit = 6
[junit4:junit4]   2> 3521 T335 C29 oass.SolrIndexSearcher.<init> Opening Searcher@3f28ad90 main
[junit4:junit4]   2> 3521 T335 C29 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3521 T335 C29 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 3523 T336 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3f28ad90 main{StandardDirectoryReader(segments_6:14 _3(4.1):C1)}
[junit4:junit4]   2> 3524 T335 C29 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 59
[junit4:junit4]   2> 3527 T334 C29 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 3532 T335 C29 REQ [collection1] webapp=/solr path=/select params={wt=csv&q=*:*&fl=id} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 3553 T333 C29 REQ [collection1] webapp=/solr path=/debug/dump params={wt=xml&stream.url=http://127.0.0.1:63066/solr/select?q%3D*:*%26fl%3Did%26wt%3Dcsv} status=0 QTime=24 
[junit4:junit4]   2> 3554 T321 oas.SolrTestCaseJ4.tearDown ###Ending testStreamUrl
[junit4:junit4]   2> 3601 T321 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=601419896
[junit4:junit4]   2> 3601 T321 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@3e5d2347
[junit4:junit4]   2> 3611 T321 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=5,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=4,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4]   2> 3611 T321 oasc.SolrCore.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 3612 T321 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 3612 T321 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 3613 T321 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 3634 T321 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 3736 T321 oejut.QueuedThreadPool.doStop WARNING 4 threads could not be stopped
[junit4:junit4]   2> 3736 T321 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> 3772 T320 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 4 leaked thread(s).
[junit4:junit4]   2> 23927 T320 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming: 
[junit4:junit4]   2> 	   1) Thread[id=329, name=qtp651274847-329 Acceptor1 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   2) Thread[id=331, name=qtp651274847-331 Acceptor3 SelectChannelConnector@0.0.0.0:?????, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   3) Thread[id=330, name=qtp651274847-330 Acceptor2 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   4) Thread[id=328, name=qtp651274847-328 Acceptor0 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 23928 T320 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
[junit4:junit4]   2> 	   1) Thread[id=329, name=qtp651274847-329 Acceptor1 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	   2) Thread[id=331, name=qtp651274847-331 Acceptor3 SelectChannelConnector@0.0.0.0:?????, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	   3) Thread[id=330, name=qtp651274847-330 Acceptor2 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	   4) Thread[id=328, name=qtp651274847-328 Acceptor0 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 26971 T320 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
[junit4:junit4]   2> 	   1) Thread[id=329, name=qtp651274847-329 Acceptor1 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   2) Thread[id=331, name=qtp651274847-331 Acceptor3 SelectChannelConnector@0.0.0.0:?????, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   3) Thread[id=330, name=qtp651274847-330 Acceptor2 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   4) Thread[id=328, name=qtp651274847-328 Acceptor0 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=SimpleText, sim=DefaultSimilarity, locale=hi_IN, timezone=Canada/Yukon
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=5,free=204122560,total=394264576
[junit4:junit4]   2> NOTE: All tests run in this JVM: [SolrInfoMBeanTest, StandardRequestHandlerTest, TestUtils, TestCharFilters, TestBM25SimilarityFactory, TestSurroundQueryParser, TestStressVersions, DocumentAnalysisRequestHandlerTest, DebugComponentTest, TestRealTimeGet, TestFastOutputStream, PrimitiveFieldTypeTest, TestCodecSupport, TestUpdate, TestPropInject, TestRecovery, TestJmxIntegration, XmlUpdateRequestHandlerTest, TestFunctionQuery, ZkCLITest, TermVectorComponentTest, MultiTermTest, EchoParamsTest, SolrCoreTest, TestRemoteStreaming]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestRemoteStreaming -Dtests.seed=C6F54FC7A3B9989F -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=hi_IN -Dtests.timezone=Canada/Yukon -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR   0.00s J1 | TestRemoteStreaming (suite) <<<
[junit4:junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming: 
[junit4:junit4]    >    1) Thread[id=329, name=qtp651274847-329 Acceptor1 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=331, name=qtp651274847-331 Acceptor3 SelectChannelConnector@0.0.0.0:?????, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=330, name=qtp651274847-330 Acceptor2 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=328, name=qtp651274847-328 Acceptor0 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([C6F54FC7A3B9989F]:0)
[junit4:junit4]    > Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4]    >    1) Thread[id=329, name=qtp651274847-329 Acceptor1 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=331, name=qtp651274847-331 Acceptor3 SelectChannelConnector@0.0.0.0:?????, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=330, name=qtp651274847-330 Acceptor2 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=328, name=qtp651274847-328 Acceptor0 SelectChannelConnector@0.0.0.0:?????, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([C6F54FC7A3B9989F]:0)
[junit4:junit4] Completed on J1 in 27.19s, 4 tests, 2 errors <<< FAILURES!

[...truncated 452 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4]   2> 4 T99 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1352218906704
[junit4:junit4]   2> 10 T99 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 103 T100 oaz.Environment.logEnv Server environment:zookeeper.version=3.3.6-1366786, built on 07/29/2012 06:22 GMT
[junit4:junit4]   2> 104 T100 oaz.Environment.logEnv Server environment:host.name=lucene.zones.apache.org
[junit4:junit4]   2> 104 T100 oaz.Environment.logEnv Server environment:java.version=1.7.0_09
[junit4:junit4]   2> 104 T100 oaz.Environment.logEnv Server environment:java.vendor=Oracle Corporation
[junit4:junit4]   2> 104 T100 oaz.Environment.logEnv Server environment:java.home=/usr/local/openjdk7/jre
[junit4:junit4]   2> 104 T100 oaz.Environment.logEnv Server environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/common/lucene-analyzers-common-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/highlighter/lucene-highlighter-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/memory/lucene-memory-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/misc/lucene-misc-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/spatial/lucene-spatial-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/suggest/lucene-suggest-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/grouping/lucene-grouping-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queries/lucene-queries-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queryparser/lucene-queryparser-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/easymock-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/guava-r05.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/log4j-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/zookeeper-3.3.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-continuation-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-deploy-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-http-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-io-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-jmx-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-security-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-server-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-servlet-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-util-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-webapp-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-xml-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-javamail.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jdepend.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.4.jar
[junit4:junit4]   2> 105 T100 oaz.Environment.logEnv Server environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
[junit4:junit4]   2> 105 T100 oaz.Environment.logEnv Server environment:java.io.tmpdir=.
[junit4:junit4]   2> 106 T100 oaz.Environment.logEnv Server environment:java.compiler=<NA>
[junit4:junit4]   2> 106 T100 oaz.Environment.logEnv Server environment:os.name=FreeBSD
[junit4:junit4]   2> 106 T100 oaz.Environment.logEnv Server environment:os.arch=amd64
[junit4:junit4]   2> 106 T100 oaz.Environment.logEnv Server environment:os.version=9.0-RELEASE
[junit4:junit4]   2> 106 T100 oaz.Environment.logEnv Server environment:user.name=hudson
[junit4:junit4]   2> 106 T100 oaz.Environment.logEnv Server environment:user.home=/home/hudson
[junit4:junit4]   2> 107 T100 oaz.Environment.logEnv Server environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 143 T100 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 173 T100 oazs.NIOServerCnxn$Factory.<init> binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 218 T99 oasc.ZkTestServer.run start zk server on port:64775
[junit4:junit4]   2> 236 T100 oazsp.FileTxnSnapLog.save Snapshotting: 0
[junit4:junit4]   2> 308 T99 oaz.Environment.logEnv Client environment:zookeeper.version=3.3.6-1366786, built on 07/29/2012 06:22 GMT
[junit4:junit4]   2> 308 T99 oaz.Environment.logEnv Client environment:host.name=lucene.zones.apache.org
[junit4:junit4]   2> 309 T99 oaz.Environment.logEnv Client environment:java.version=1.7.0_09
[junit4:junit4]   2> 309 T99 oaz.Environment.logEnv Client environment:java.vendor=Oracle Corporation
[junit4:junit4]   2> 309 T99 oaz.Environment.logEnv Client environment:java.home=/usr/local/openjdk7/jre
[junit4:junit4]   2> 310 T99 oaz.Environment.logEnv Client environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/common/lucene-analyzers-common-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/highlighter/lucene-highlighter-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/memory/lucene-memory-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/misc/lucene-misc-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/spatial/lucene-spatial-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/suggest/lucene-suggest-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/grouping/lucene-grouping-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queries/lucene-queries-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queryparser/lucene-queryparser-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/easymock-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/guava-r05.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/log4j-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/zookeeper-3.3.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-continuation-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-deploy-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-http-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-io-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-jmx-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-security-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-server-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-servlet-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-util-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-webapp-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-xml-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-javamail.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jdepend.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.4.jar
[junit4:junit4]   2> 311 T99 oaz.Environment.logEnv Client environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
[junit4:junit4]   2> 311 T99 oaz.Environment.logEnv Client environment:java.io.tmpdir=.
[junit4:junit4]   2> 311 T99 oaz.Environment.logEnv Client environment:java.compiler=<NA>
[junit4:junit4]   2> 312 T99 oaz.Environment.logEnv Client environment:os.name=FreeBSD
[junit4:junit4]   2> 312 T99 oaz.Environment.logEnv Client environment:os.arch=amd64
[junit4:junit4]   2> 312 T99 oaz.Environment.logEnv Client environment:os.version=9.0-RELEASE
[junit4:junit4]   2> 313 T99 oaz.Environment.logEnv Client environment:user.name=hudson
[junit4:junit4]   2> 313 T99 oaz.Environment.logEnv Client environment:user.home=/home/hudson
[junit4:junit4]   2> 313 T99 oaz.Environment.logEnv Client environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 319 T99 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:64775 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@5d1ae1c3
[junit4:junit4]   2> 374 T104 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:64775
[junit4:junit4]   2> 375 T99 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 378 T104 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:64775, initiating session
[junit4:junit4]   2> 378 T101 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:51518
[junit4:junit4]   2> 394 T101 oazs.NIOServerCnxn.doIO WARNING Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
[junit4:junit4]   2> 394 T101 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:51518 (no session established for client)
[junit4:junit4]   2> 395 T104 oaz.ClientCnxn$SendThread.run Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
[junit4:junit4]   2> 2127 T104 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost/127.0.0.1:64775
[junit4:junit4]   2> 2128 T104 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:64775, initiating session
[junit4:junit4]   2> 2128 T101 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:21267
[junit4:junit4]   2> 2129 T101 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:21267
[junit4:junit4]   2> 2144 T103 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 2176 T103 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad687116a0000 with negotiated timeout 10000 for client /140.211.11.196:21267
[junit4:junit4]   2> 2176 T104 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:64775, sessionid = 0x13ad687116a0000, negotiated timeout = 10000
[junit4:junit4]   2> 2180 T106 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5d1ae1c3 name:ZooKeeperConnection Watcher:127.0.0.1:64775 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2180 T99 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2185 T99 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 2252 T105 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x13ad687116a0000
[junit4:junit4]   2> 2253 T106 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 2253 T99 oaz.ZooKeeper.close Session: 0x13ad687116a0000 closed
[junit4:junit4]   2> 2254 T99 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:64775/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@13c77dd0
[junit4:junit4]   2> 2254 T101 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:21267 which had sessionid 0x13ad687116a0000
[junit4:junit4]   2> 2255 T107 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:64775
[junit4:junit4]   2> 2255 T99 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2256 T107 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:64775, initiating session
[junit4:junit4]   2> 2256 T101 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:35059
[junit4:junit4]   2> 2256 T101 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:35059
[junit4:junit4]   2> 2258 T103 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad687116a0001 with negotiated timeout 10000 for client /140.211.11.196:35059
[junit4:junit4]   2> 2258 T107 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:64775, sessionid = 0x13ad687116a0001, negotiated timeout = 10000
[junit4:junit4]   2> 2258 T108 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13c77dd0 name:ZooKeeperConnection Watcher:127.0.0.1:64775/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2259 T99 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2268 T99 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 2273 T99 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 2283 T99 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 2286 T99 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 2289 T99 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2318 T99 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2336 T99 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 2337 T99 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 2442 T99 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2443 T99 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2481 T99 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 2481 T99 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 2486 T99 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 2486 T99 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 2489 T99 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 2490 T99 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 2494 T99 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 2494 T99 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 2497 T99 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 2499 T99 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 2502 T99 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 2502 T99 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 2505 T99 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 2506 T99 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 2509 T105 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x13ad687116a0001
[junit4:junit4]   2> 2517 T108 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 2517 T101 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:35059 which had sessionid 0x13ad687116a0001
[junit4:junit4]   2> 2517 T99 oaz.ZooKeeper.close Session: 0x13ad687116a0001 closed
[junit4:junit4]   2> 2980 T99 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 3245 T99 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:38672
[junit4:junit4]   2> 3277 T99 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3278 T99 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 3278 T99 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 3315 T99 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 3316 T99 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3316 T99 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 3316 T99 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 3317 T99 oasc.CoreContainer.<init> New CoreContainer 1200082580
[junit4:junit4]   2> 3317 T99 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 3318 T99 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 3363 T99 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 3384 T99 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:64775/solr
[junit4:junit4]   2> 3420 T99 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:64775/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@198cc898
[junit4:junit4]   2> 3421 T119 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:64775
[junit4:junit4]   2> 3421 T99 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3422 T119 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:64775, initiating session
[junit4:junit4]   2> 3422 T101 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:17343
[junit4:junit4]   2> 3423 T101 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:17343
[junit4:junit4]   2> 3424 T103 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad687116a0002 with negotiated timeout 5000 for client /140.211.11.196:17343
[junit4:junit4]   2> 3424 T119 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:64775, sessionid = 0x13ad687116a0002, negotiated timeout = 5000
[junit4:junit4]   2> 3425 T120 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@198cc898 name:ZooKeeperConnection Watcher:127.0.0.1:64775/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3425 T99 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3445 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0002 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3456 T99 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 3459 T99 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38672_solr
[junit4:junit4]   2> 3465 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0002 type:delete cxid:0x7 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:38672_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:38672_solr
[junit4:junit4]   2> 3466 T99 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38672_solr
[junit4:junit4]   2> 3469 T99 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 3470 T99 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 3470 T99 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 3470 T99 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 3471 T99 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 3471 T99 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 3471 T99 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 3472 T99 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 3472 T99 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 3473 T99 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 3487 T99 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 3506 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0002 type:delete cxid:0x15 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 3507 T99 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 3510 T99 oasc.Overseer.start Overseer (id=88619018288234498-127.0.0.1:38672_solr-n_0000000000) starting
[junit4:junit4]   2> 3510 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0002 type:create cxid:0x1a zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3526 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0002 type:create cxid:0x1b zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3527 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0002 type:create cxid:0x1c zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3533 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0002 type:create cxid:0x1d zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3534 T122 oasc.OverseerCollectionProcessor.run Process current queue of collection messages
[junit4:junit4]   2> 3535 T99 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 3542 T99 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 3562 T121 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 3606 T99 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 3606 T99 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 3607 T99 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 3608 T99 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 3610 T99 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 3611 T99 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 3611 T99 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 3693 T99 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 3743 T99 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 3844 T99 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 3852 T99 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 4296 T99 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 4306 T99 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 4309 T99 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 4329 T99 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4334 T99 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4339 T99 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4342 T99 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4345 T99 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/control/data/
[junit4:junit4]   2> 4345 T99 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2691cde4
[junit4:junit4]   2> 4370 T99 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/control/data/index/
[junit4:junit4]   2> 4371 T99 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 4372 T99 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/control/data/index forceNew:false
[junit4:junit4]   2> 4382 T99 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/control/data/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 4382 T99 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 4383 T99 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 4385 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 4386 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 4386 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 4387 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 4387 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 4388 T99 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 4388 T99 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 4389 T99 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 4390 T99 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 4390 T99 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 4391 T99 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 4392 T99 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 4392 T99 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 4393 T99 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 4393 T99 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 4394 T99 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 4394 T99 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4395 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4396 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4396 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4397 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4397 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4398 T99 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4399 T99 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 4399 T99 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 4408 T99 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 4412 T99 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 4413 T99 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 4417 T99 oass.SolrIndexSearcher.<init> Opening Searcher@603678ef main
[junit4:junit4]   2> 4418 T99 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 4418 T99 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 4419 T99 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 4433 T99 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 4503 T126 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@603678ef main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 4508 T99 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 4509 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0002 type:create cxid:0x53 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 4522 T99 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 4523 T99 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:38672/solr shardId:control_shard
[junit4:junit4]   2> 4532 T99 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 4546 T99 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/control_shard/election
[junit4:junit4]   2> 4573 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0002 type:delete cxid:0x64 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 4575 T99 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 4575 T121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4576 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0002 type:create cxid:0x67 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4588 T121 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"control_shard",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:38672_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:38672/solr"}
[junit4:junit4]   2> 4600 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0002 type:create cxid:0x6a zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 4644 T120 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (1)
[junit4:junit4]   2> 5097 T99 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 5097 T99 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 5098 T99 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:38672/solr/collection1/
[junit4:junit4]   2> 5099 T99 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 5099 T99 oasc.SyncStrategy.syncToMe http://127.0.0.1:38672/solr/collection1/ has no replicas
[junit4:junit4]   2> 5100 T99 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:38672/solr/collection1/
[junit4:junit4]   2> 5101 T99 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/control_shard
[junit4:junit4]   2> 5112 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0002 type:create cxid:0x88 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5157 T121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5170 T120 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (1)
[junit4:junit4]   2> 5229 T99 oasc.ZkController.register We are http://127.0.0.1:38672/solr/collection1/ and leader is http://127.0.0.1:38672/solr/collection1/
[junit4:junit4]   2> 5229 T99 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:38672/solr
[junit4:junit4]   2> 5230 T99 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 5230 T99 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 5233 T99 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5234 T99 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 5235 T99 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 5292 T99 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 5440 T99 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 5443 T99 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:19972
[junit4:junit4]   2> 5443 T99 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 5444 T99 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352218912010
[junit4:junit4]   2> 5444 T99 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352218912010/'
[junit4:junit4]   2> 5482 T99 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 5482 T99 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 5483 T99 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352218912010
[junit4:junit4]   2> 5483 T99 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352218912010/solr.xml
[junit4:junit4]   2> 5483 T99 oasc.CoreContainer.<init> New CoreContainer 1719203423
[junit4:junit4]   2> 5484 T99 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352218912010/'
[junit4:junit4]   2> 5484 T99 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352218912010/'
[junit4:junit4]   2> 5508 T99 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 5519 T99 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:64775/solr
[junit4:junit4]   2> 5520 T99 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:64775/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@45d5584a
[junit4:junit4]   2> 5521 T136 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:64775
[junit4:junit4]   2> 5521 T99 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 5522 T136 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:64775, initiating session
[junit4:junit4]   2> 5522 T101 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:24932
[junit4:junit4]   2> 5523 T101 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:24932
[junit4:junit4]   2> 5533 T103 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad687116a0003 with negotiated timeout 5000 for client /140.211.11.196:24932
[junit4:junit4]   2> 5533 T136 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:64775, sessionid = 0x13ad687116a0003, negotiated timeout = 5000
[junit4:junit4]   2> 5534 T137 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@45d5584a name:ZooKeeperConnection Watcher:127.0.0.1:64775/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 5534 T99 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 5535 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0003 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5537 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0003 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5546 T99 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:19972_solr
[junit4:junit4]   2> 5547 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0003 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:19972_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:19972_solr
[junit4:junit4]   2> 5548 T99 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:19972_solr
[junit4:junit4]   2> 5558 T120 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 5559 T99 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 5559 T99 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 5559 T99 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 5560 T99 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 5560 T99 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 5561 T99 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 5561 T99 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 5561 T99 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 5562 T99 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 5562 T99 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 5583 T99 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 5589 T99 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352218912010/collection1
[junit4:junit4]   2> 5589 T99 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 5590 T99 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 5591 T99 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 5592 T99 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352218912010/collection1/'
[junit4:junit4]   2> 5593 T99 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty1-1352218912010/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 5594 T99 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty1-1352218912010/collection1/lib/README' to classloader
[junit4:junit4]   2> 5637 T99 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 5681 T121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5683 T121 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"control_shard",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:38672_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:38672/solr"}
[junit4:junit4]   2> 5688 T99 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 5790 T120 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 5815 T137 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 5815 T99 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 5823 T99 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 6261 T99 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 6270 T99 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 6273 T99 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 6289 T99 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6302 T99 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6307 T99 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6309 T99 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6313 T99 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352218912010/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty1/
[junit4:junit4]   2> 6313 T99 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2691cde4
[junit4:junit4]   2> 6314 T99 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty1/index/
[junit4:junit4]   2> 6315 T99 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 6316 T99 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty1/index forceNew:false
[junit4:junit4]   2> 6329 T99 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty1/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 6330 T99 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 6331 T99 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 6333 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 6333 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 6334 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 6334 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 6335 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 6335 T99 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 6336 T99 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 6337 T99 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 6337 T99 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 6338 T99 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 6339 T99 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 6339 T99 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 6340 T99 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 6340 T99 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 6341 T99 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 6341 T99 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 6342 T99 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6343 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6343 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6344 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6345 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6345 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6346 T99 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6347 T99 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 6347 T99 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 6356 T99 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 6360 T99 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 6361 T99 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 6364 T99 oass.SolrIndexSearcher.<init> Opening Searcher@7b0e6f6e main
[junit4:junit4]   2> 6365 T99 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 6365 T99 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 6366 T99 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 6381 T99 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 6434 T142 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7b0e6f6e main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 6439 T99 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6861 T121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6862 T121 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:19972_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:19972/solr"}
[junit4:junit4]   2> 6862 T121 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 6885 T120 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 6885 T137 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 6942 T99 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 6942 T99 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:19972/solr shardId:shard1
[junit4:junit4]   2> 6944 T99 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 6964 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0003 type:delete cxid:0x43 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 6965 T99 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 6966 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0003 type:create cxid:0x44 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6977 T99 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 6977 T99 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 6978 T99 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:19972/solr/collection1/
[junit4:junit4]   2> 6978 T99 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 6979 T99 oasc.SyncStrategy.syncToMe http://127.0.0.1:19972/solr/collection1/ has no replicas
[junit4:junit4]   2> 6980 T99 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:19972/solr/collection1/
[junit4:junit4]   2> 6980 T99 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 6986 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0003 type:create cxid:0x4e zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7391 T121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 7432 T120 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 7432 T137 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 7454 T99 oasc.ZkController.register We are http://127.0.0.1:19972/solr/collection1/ and leader is http://127.0.0.1:19972/solr/collection1/
[junit4:junit4]   2> 7454 T99 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:19972/solr
[junit4:junit4]   2> 7455 T99 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 7455 T99 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 7462 T99 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 7463 T99 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 7464 T99 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 7464 T99 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 7590 T99 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 7593 T99 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:64816
[junit4:junit4]   2> 7594 T99 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 7594 T99 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352218914175
[junit4:junit4]   2> 7594 T99 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352218914175/'
[junit4:junit4]   2> 7624 T99 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 7625 T99 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 7625 T99 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352218914175
[junit4:junit4]   2> 7625 T99 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352218914175/solr.xml
[junit4:junit4]   2> 7626 T99 oasc.CoreContainer.<init> New CoreContainer 764890953
[junit4:junit4]   2> 7626 T99 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352218914175/'
[junit4:junit4]   2> 7627 T99 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352218914175/'
[junit4:junit4]   2> 7651 T99 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 7662 T99 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:64775/solr
[junit4:junit4]   2> 7663 T99 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:64775/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@579af7c4
[junit4:junit4]   2> 7664 T152 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:64775
[junit4:junit4]   2> 7664 T99 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7665 T152 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:64775, initiating session
[junit4:junit4]   2> 7666 T101 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:36848
[junit4:junit4]   2> 7666 T101 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:36848
[junit4:junit4]   2> 7668 T103 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad687116a0004 with negotiated timeout 5000 for client /140.211.11.196:36848
[junit4:junit4]   2> 7668 T152 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:64775, sessionid = 0x13ad687116a0004, negotiated timeout = 5000
[junit4:junit4]   2> 7668 T153 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@579af7c4 name:ZooKeeperConnection Watcher:127.0.0.1:64775/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7669 T99 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7670 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0004 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7679 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0004 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7687 T99 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64816_solr
[junit4:junit4]   2> 7688 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0004 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:64816_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:64816_solr
[junit4:junit4]   2> 7689 T99 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:64816_solr
[junit4:junit4]   2> 7693 T137 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7693 T120 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7693 T99 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 7694 T99 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 7694 T99 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 7695 T99 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 7695 T99 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 7695 T99 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 7696 T99 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 7696 T99 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 7696 T99 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 7697 T99 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 7714 T99 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 7720 T99 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352218914175/collection1
[junit4:junit4]   2> 7720 T99 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 7721 T99 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 7721 T99 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 7723 T99 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352218914175/collection1/'
[junit4:junit4]   2> 7724 T99 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty2-1352218914175/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 7724 T99 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty2-1352218914175/collection1/lib/README' to classloader
[junit4:junit4]   2> 7763 T99 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 7831 T99 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 7932 T99 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 7939 T99 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 7946 T121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 7947 T121 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:19972_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:19972/solr"}
[junit4:junit4]   2> 7961 T137 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 7961 T120 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 7961 T153 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 8386 T99 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 8395 T99 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 8398 T99 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 8414 T99 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8419 T99 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8423 T99 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8425 T99 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8427 T99 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352218914175/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty2/
[junit4:junit4]   2> 8427 T99 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2691cde4
[junit4:junit4]   2> 8428 T99 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty2/index/
[junit4:junit4]   2> 8429 T99 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 8430 T99 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty2/index forceNew:false
[junit4:junit4]   2> 8436 T99 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty2/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 8436 T99 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 8437 T99 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 8440 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 8441 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 8442 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 8442 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 8443 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 8444 T99 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 8445 T99 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 8446 T99 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 8447 T99 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 8448 T99 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 8449 T99 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 8450 T99 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 8451 T99 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 8451 T99 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 8452 T99 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 8453 T99 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 8454 T99 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8455 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8456 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8457 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8458 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8458 T99 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8459 T99 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8460 T99 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 8460 T99 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 8471 T99 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 8475 T99 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 8477 T99 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 8480 T99 oass.SolrIndexSearcher.<init> Opening Searcher@604f05f8 main
[junit4:junit4]   2> 8482 T99 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 8482 T99 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 8483 T99 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 8505 T99 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 8559 T158 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@604f05f8 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 8567 T99 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8969 T121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8971 T121 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:64816_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:64816/solr"}
[junit4:junit4]   2> 8985 T120 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 8985 T153 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 8985 T137 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 9071 T99 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 9071 T99 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:64816/solr shardId:shard2
[junit4:junit4]   2> 9073 T99 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 9099 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0004 type:delete cxid:0x42 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 9100 T99 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 9101 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0004 type:create cxid:0x43 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9111 T99 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 9112 T99 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 9112 T99 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:64816/solr/collection1/
[junit4:junit4]   2> 9113 T99 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 9113 T99 oasc.SyncStrategy.syncToMe http://127.0.0.1:64816/solr/collection1/ has no replicas
[junit4:junit4]   2> 9114 T99 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:64816/solr/collection1/
[junit4:junit4]   2> 9114 T99 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 9118 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0004 type:create cxid:0x4c zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9491 T121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9519 T120 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 9519 T153 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 9519 T137 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 9537 T99 oasc.ZkController.register We are http://127.0.0.1:64816/solr/collection1/ and leader is http://127.0.0.1:64816/solr/collection1/
[junit4:junit4]   2> 9537 T99 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:64816/solr
[junit4:junit4]   2> 9538 T99 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 9538 T99 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9540 T99 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9541 T99 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 9541 T99 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 9542 T99 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9673 T99 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 9676 T99 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:18910
[junit4:junit4]   2> 9677 T99 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 9677 T99 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352218916255
[junit4:junit4]   2> 9678 T99 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352218916255/'
[junit4:junit4]   2> 9719 T99 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 9719 T99 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 9720 T99 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352218916255
[junit4:junit4]   2> 9720 T99 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352218916255/solr.xml
[junit4:junit4]   2> 9720 T99 oasc.CoreContainer.<init> New CoreContainer 1780142497
[junit4:junit4]   2> 9721 T99 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352218916255/'
[junit4:junit4]   2> 9721 T99 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352218916255/'
[junit4:junit4]   2> 9749 T99 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 9762 T99 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:64775/solr
[junit4:junit4]   2> 9763 T99 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:64775/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@699ae127
[junit4:junit4]   2> 9764 T168 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:64775
[junit4:junit4]   2> 9763 T99 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 9765 T168 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:64775, initiating session
[junit4:junit4]   2> 9765 T101 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:18369
[junit4:junit4]   2> 9765 T101 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:18369
[junit4:junit4]   2> 9769 T103 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad687116a0005 with negotiated timeout 5000 for client /140.211.11.196:18369
[junit4:junit4]   2> 9769 T168 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:64775, sessionid = 0x13ad687116a0005, negotiated timeout = 5000
[junit4:junit4]   2> 9770 T169 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@699ae127 name:ZooKeeperConnection Watcher:127.0.0.1:64775/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 9770 T99 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 9771 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0005 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9772 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0005 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9774 T99 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:18910_solr
[junit4:junit4]   2> 9774 T105 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad687116a0005 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:18910_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:18910_solr
[junit4:junit4]   2> 9783 T99 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:18910_solr
[junit4:junit4]   2> 9786 T137 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 9786 T120 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 9786 T153 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 9787 T99 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 9787 T99 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 9787 T99 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 9788 T99 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 9788 T99 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 9788 T99 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 9789 T99 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 9789 T99 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 9789 T99 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 9790 T99 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 9806 T99 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 9812 T99 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352218916255/collection1
[junit4:junit4]   2> 9813 T99 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 9813 T99 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 9814 T99 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 9815 T99 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352218916255/collection1/'
[junit4:junit4]   2> 9816 T99 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty3-1352218916255/collection1/lib/README' to classloader
[junit4:junit4]   2> 9817 T99 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty3-1352218916255/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 9862 T99 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 9917 T99 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 10018 T99 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 10026 T121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 10027 T121 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:64816_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:64816/solr"}
[junit4:junit4]   2> 10029 T99 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 10086 T137 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 10086 T120 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 10086 T169 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 10086 T153 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 10573 T99 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 10583 T99 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 10586 T99 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 10610 T99 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10618 T99 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10624 T99 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10626 T99 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10629 T99 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352218916255/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty3/
[junit4:junit4]   2> 10629 T99 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2691cde4
[junit4:junit4]   2> 10631 T99 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty3/index/
[junit4:junit4]   2> 10632 T99 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 10634 T99 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty3/index forceNew:false
[junit4:junit4]   2> 10641 T99 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1352218906703/jetty3/index,segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 10642 T99 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 10643 T99 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 10646 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 10647 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 10647 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 10648 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 10649 T99 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 10650 T99 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 10651 T99 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 10652 T99 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 10653 T99 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 10654 T99 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 10655 T99 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr

[...truncated too long message...]

r.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=117, name=qtp1065037894-117 Acceptor2 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=165, name=qtp15205627-165 Acceptor1 SelectChannelConnector@0.0.0.0:18910, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    5) Thread[id=186, name=qtp507512929-186 Acceptor3 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    6) Thread[id=167, name=qtp15205627-167 Acceptor3 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    7) Thread[id=115, name=qtp1065037894-115 Acceptor0 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    8) Thread[id=184, name=qtp507512929-184 Acceptor1 SelectChannelConnector@0.0.0.0:24328, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    9) Thread[id=116, name=qtp1065037894-116 Acceptor1 SelectChannelConnector@0.0.0.0:38672, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   10) Thread[id=164, name=qtp15205627-164 Acceptor0 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   11) Thread[id=118, name=qtp1065037894-118 Acceptor3 SelectChannelConnector@0.0.0.0:38672, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   12) Thread[id=166, name=qtp15205627-166 Acceptor2 SelectChannelConnector@0.0.0.0:18910, state=BLOCKED, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([C6F54FC7A3B9989F]:0)
[junit4:junit4] Completed on J0 in 292.06s, 1 test, 2 errors <<< FAILURES!

[...truncated 149 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:335: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/module-build.xml:61: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:1163: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:827: There were test failures: 233 suites, 965 tests, 4 suite-level errors, 744 ignored (1 assumption)

Total time: 22 minutes 59 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 612 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/612/

5 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZk2Test

Error Message:
16 threads leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZk2Test:     1) Thread[id=96, name=qtp1946791897-96 Acceptor1 SelectChannelConnector@0.0.0.0:32207, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=47, name=qtp526921047-47 Acceptor2 SelectChannelConnector@0.0.0.0:54171, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=30, name=qtp399389739-30 Acceptor3 SelectChannelConnector@0.0.0.0:23671, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=97, name=qtp1946791897-97 Acceptor2 SelectChannelConnector@0.0.0.0:32207, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=95, name=qtp1946791897-95 Acceptor0 SelectChannelConnector@0.0.0.0:32207, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=79, name=qtp1070454762-79 Acceptor2 SelectChannelConnector@0.0.0.0:37521, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=77, name=qtp1070454762-77 Acceptor0 SelectChannelConnector@0.0.0.0:37521, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=29, name=qtp399389739-29 Acceptor2 SelectChannelConnector@0.0.0.0:23671, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=98, name=qtp1946791897-98 Acceptor3 SelectChannelConnector@0.0.0.0:32207, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=80, name=qtp1070454762-80 Acceptor3 SelectChannelConnector@0.0.0.0:37521, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=78, name=qtp1070454762-78 Acceptor1 SelectChannelConnector@0.0.0.0:37521, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=45, name=qtp526921047-45 Acceptor0 SelectChannelConnector@0.0.0.0:54171, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=28, name=qtp399389739-28 Acceptor1 SelectChannelConnector@0.0.0.0:23671, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=48, name=qtp526921047-48 Acceptor3 SelectChannelConnector@0.0.0.0:54171, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=46, name=qtp526921047-46 Acceptor1 SelectChannelConnector@0.0.0.0:54171, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=27, name=qtp399389739-27 Acceptor0 SelectChannelConnector@0.0.0.0:23671, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 16 threads leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZk2Test: 
   1) Thread[id=96, name=qtp1946791897-96 Acceptor1 SelectChannelConnector@0.0.0.0:32207, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=47, name=qtp526921047-47 Acceptor2 SelectChannelConnector@0.0.0.0:54171, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=30, name=qtp399389739-30 Acceptor3 SelectChannelConnector@0.0.0.0:23671, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=97, name=qtp1946791897-97 Acceptor2 SelectChannelConnector@0.0.0.0:32207, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=95, name=qtp1946791897-95 Acceptor0 SelectChannelConnector@0.0.0.0:32207, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=79, name=qtp1070454762-79 Acceptor2 SelectChannelConnector@0.0.0.0:37521, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=77, name=qtp1070454762-77 Acceptor0 SelectChannelConnector@0.0.0.0:37521, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=29, name=qtp399389739-29 Acceptor2 SelectChannelConnector@0.0.0.0:23671, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=98, name=qtp1946791897-98 Acceptor3 SelectChannelConnector@0.0.0.0:32207, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=80, name=qtp1070454762-80 Acceptor3 SelectChannelConnector@0.0.0.0:37521, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=78, name=qtp1070454762-78 Acceptor1 SelectChannelConnector@0.0.0.0:37521, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=45, name=qtp526921047-45 Acceptor0 SelectChannelConnector@0.0.0.0:54171, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=28, name=qtp399389739-28 Acceptor1 SelectChannelConnector@0.0.0.0:23671, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=48, name=qtp526921047-48 Acceptor3 SelectChannelConnector@0.0.0.0:54171, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=46, name=qtp526921047-46 Acceptor1 SelectChannelConnector@0.0.0.0:54171, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=27, name=qtp399389739-27 Acceptor0 SelectChannelConnector@0.0.0.0:23671, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([3AA65C7CB87FC8FA]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZk2Test

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=47, name=qtp526921047-47 Acceptor2 SelectChannelConnector@0.0.0.0:54171, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=96, name=qtp1946791897-96 Acceptor1 SelectChannelConnector@0.0.0.0:32207, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=30, name=qtp399389739-30 Acceptor3 SelectChannelConnector@0.0.0.0:23671, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=97, name=qtp1946791897-97 Acceptor2 SelectChannelConnector@0.0.0.0:32207, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=79, name=qtp1070454762-79 Acceptor2 SelectChannelConnector@0.0.0.0:37521, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=95, name=qtp1946791897-95 Acceptor0 SelectChannelConnector@0.0.0.0:32207, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=77, name=qtp1070454762-77 Acceptor0 SelectChannelConnector@0.0.0.0:37521, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=29, name=qtp399389739-29 Acceptor2 SelectChannelConnector@0.0.0.0:23671, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=98, name=qtp1946791897-98 Acceptor3 SelectChannelConnector@0.0.0.0:32207, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=80, name=qtp1070454762-80 Acceptor3 SelectChannelConnector@0.0.0.0:37521, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=78, name=qtp1070454762-78 Acceptor1 SelectChannelConnector@0.0.0.0:37521, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=45, name=qtp526921047-45 Acceptor0 SelectChannelConnector@0.0.0.0:54171, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=28, name=qtp399389739-28 Acceptor1 SelectChannelConnector@0.0.0.0:23671, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=48, name=qtp526921047-48 Acceptor3 SelectChannelConnector@0.0.0.0:54171, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=46, name=qtp526921047-46 Acceptor1 SelectChannelConnector@0.0.0.0:54171, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=27, name=qtp399389739-27 Acceptor0 SelectChannelConnector@0.0.0.0:23671, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=47, name=qtp526921047-47 Acceptor2 SelectChannelConnector@0.0.0.0:54171, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=96, name=qtp1946791897-96 Acceptor1 SelectChannelConnector@0.0.0.0:32207, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=30, name=qtp399389739-30 Acceptor3 SelectChannelConnector@0.0.0.0:23671, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=97, name=qtp1946791897-97 Acceptor2 SelectChannelConnector@0.0.0.0:32207, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=79, name=qtp1070454762-79 Acceptor2 SelectChannelConnector@0.0.0.0:37521, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=95, name=qtp1946791897-95 Acceptor0 SelectChannelConnector@0.0.0.0:32207, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=77, name=qtp1070454762-77 Acceptor0 SelectChannelConnector@0.0.0.0:37521, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=29, name=qtp399389739-29 Acceptor2 SelectChannelConnector@0.0.0.0:23671, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=98, name=qtp1946791897-98 Acceptor3 SelectChannelConnector@0.0.0.0:32207, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=80, name=qtp1070454762-80 Acceptor3 SelectChannelConnector@0.0.0.0:37521, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=78, name=qtp1070454762-78 Acceptor1 SelectChannelConnector@0.0.0.0:37521, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=45, name=qtp526921047-45 Acceptor0 SelectChannelConnector@0.0.0.0:54171, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=28, name=qtp399389739-28 Acceptor1 SelectChannelConnector@0.0.0.0:23671, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=48, name=qtp526921047-48 Acceptor3 SelectChannelConnector@0.0.0.0:54171, state=RUNNABLE, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=46, name=qtp526921047-46 Acceptor1 SelectChannelConnector@0.0.0.0:54171, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=27, name=qtp399389739-27 Acceptor0 SelectChannelConnector@0.0.0.0:23671, state=BLOCKED, group=TGRP-BasicDistributedZk2Test]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([3AA65C7CB87FC8FA]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([3AA65C7CB87FC8FA]:0)


FAILED:  org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch

Error Message:
Shard still reported as live in zk

Stack Trace:
java.lang.AssertionError: Shard still reported as live in zk
	at __randomizedtesting.SeedInfo.seed([3AA65C7CB87FC8FA:BB40D264CF20A8C6]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitToSeeNotLive(AbstractFullDistribZkTestBase.java:1218)
	at org.apache.solr.cloud.BasicDistributedZk2Test.brindDownShardIndexSomeDocsAndRecover(BasicDistributedZk2Test.java:190)
	at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:83)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:695)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)


FAILED:  org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testDistribSearch

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([3AA65C7CB87FC8FA]:0)




Build Log:
[...truncated 8309 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
[junit4:junit4]   2> 233 T11 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1352168085568
[junit4:junit4]   2> 248 T11 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 347 T12 oaz.Environment.logEnv Server environment:zookeeper.version=3.3.6-1366786, built on 07/29/2012 06:22 GMT
[junit4:junit4]   2> 349 T12 oaz.Environment.logEnv Server environment:host.name=lucene.zones.apache.org
[junit4:junit4]   2> 349 T12 oaz.Environment.logEnv Server environment:java.version=1.7.0_09
[junit4:junit4]   2> 350 T12 oaz.Environment.logEnv Server environment:java.vendor=Oracle Corporation
[junit4:junit4]   2> 350 T12 oaz.Environment.logEnv Server environment:java.home=/usr/local/openjdk7/jre
[junit4:junit4]   2> 351 T12 oaz.Environment.logEnv Server environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/common/lucene-analyzers-common-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/highlighter/lucene-highlighter-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/memory/lucene-memory-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/misc/lucene-misc-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/spatial/lucene-spatial-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/suggest/lucene-suggest-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/grouping/lucene-grouping-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queries/lucene-queries-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queryparser/lucene-queryparser-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/easymock-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/guava-r05.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/log4j-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/zookeeper-3.3.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-continuation-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-deploy-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-http-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-io-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-jmx-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-security-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-server-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-servlet-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-util-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-webapp-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-xml-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-javamail.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jdepend.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.4.jar
[junit4:junit4]   2> 355 T12 oaz.Environment.logEnv Server environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
[junit4:junit4]   2> 356 T12 oaz.Environment.logEnv Server environment:java.io.tmpdir=.
[junit4:junit4]   2> 357 T12 oaz.Environment.logEnv Server environment:java.compiler=<NA>
[junit4:junit4]   2> 357 T12 oaz.Environment.logEnv Server environment:os.name=FreeBSD
[junit4:junit4]   2> 358 T12 oaz.Environment.logEnv Server environment:os.arch=amd64
[junit4:junit4]   2> 358 T12 oaz.Environment.logEnv Server environment:os.version=9.0-RELEASE
[junit4:junit4]   2> 359 T12 oaz.Environment.logEnv Server environment:user.name=hudson
[junit4:junit4]   2> 359 T12 oaz.Environment.logEnv Server environment:user.home=/home/hudson
[junit4:junit4]   2> 360 T12 oaz.Environment.logEnv Server environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 397 T12 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 428 T12 oazs.NIOServerCnxn$Factory.<init> binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 457 T11 oasc.ZkTestServer.run start zk server on port:33140
[junit4:junit4]   2> 490 T12 oazsp.FileTxnSnapLog.save Snapshotting: 0
[junit4:junit4]   2> 523 T11 oaz.Environment.logEnv Client environment:zookeeper.version=3.3.6-1366786, built on 07/29/2012 06:22 GMT
[junit4:junit4]   2> 524 T11 oaz.Environment.logEnv Client environment:host.name=lucene.zones.apache.org
[junit4:junit4]   2> 525 T11 oaz.Environment.logEnv Client environment:java.version=1.7.0_09
[junit4:junit4]   2> 526 T11 oaz.Environment.logEnv Client environment:java.vendor=Oracle Corporation
[junit4:junit4]   2> 526 T11 oaz.Environment.logEnv Client environment:java.home=/usr/local/openjdk7/jre
[junit4:junit4]   2> 527 T11 oaz.Environment.logEnv Client environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/common/lucene-analyzers-common-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/highlighter/lucene-highlighter-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/memory/lucene-memory-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/misc/lucene-misc-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/spatial/lucene-spatial-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/suggest/lucene-suggest-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/grouping/lucene-grouping-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queries/lucene-queries-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queryparser/lucene-queryparser-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/easymock-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/guava-r05.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/log4j-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/zookeeper-3.3.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-continuation-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-deploy-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-http-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-io-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-jmx-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-security-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-server-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-servlet-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-util-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-webapp-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-xml-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-javamail.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jdepend.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.4.jar
[junit4:junit4]   2> 530 T11 oaz.Environment.logEnv Client environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
[junit4:junit4]   2> 530 T11 oaz.Environment.logEnv Client environment:java.io.tmpdir=.
[junit4:junit4]   2> 531 T11 oaz.Environment.logEnv Client environment:java.compiler=<NA>
[junit4:junit4]   2> 532 T11 oaz.Environment.logEnv Client environment:os.name=FreeBSD
[junit4:junit4]   2> 532 T11 oaz.Environment.logEnv Client environment:os.arch=amd64
[junit4:junit4]   2> 533 T11 oaz.Environment.logEnv Client environment:os.version=9.0-RELEASE
[junit4:junit4]   2> 534 T11 oaz.Environment.logEnv Client environment:user.name=hudson
[junit4:junit4]   2> 534 T11 oaz.Environment.logEnv Client environment:user.home=/home/hudson
[junit4:junit4]   2> 535 T11 oaz.Environment.logEnv Client environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 539 T11 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33140 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@3fafe567
[junit4:junit4]   2> 581 T16 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:33140
[junit4:junit4]   2> 582 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 587 T16 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33140, initiating session
[junit4:junit4]   2> 588 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:15787
[junit4:junit4]   2> 605 T13 oazs.NIOServerCnxn.doIO WARNING Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
[junit4:junit4]   2> 606 T13 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:15787 (no session established for client)
[junit4:junit4]   2> 607 T16 oaz.ClientCnxn$SendThread.run Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
[junit4:junit4]   2> 1878 T16 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost/127.0.0.1:33140
[junit4:junit4]   2> 1879 T16 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33140, initiating session
[junit4:junit4]   2> 1879 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:35504
[junit4:junit4]   2> 1881 T13 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:35504
[junit4:junit4]   2> 1900 T15 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 1984 T15 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad37f995c0000 with negotiated timeout 10000 for client /140.211.11.196:35504
[junit4:junit4]   2> 1984 T16 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:33140, sessionid = 0x13ad37f995c0000, negotiated timeout = 10000
[junit4:junit4]   2> 1990 T18 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3fafe567 name:ZooKeeperConnection Watcher:127.0.0.1:33140 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1991 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1996 T11 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 2116 T17 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x13ad37f995c0000
[junit4:junit4]   2> 2122 T18 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 2122 T11 oaz.ZooKeeper.close Session: 0x13ad37f995c0000 closed
[junit4:junit4]   2> 2123 T13 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:35504 which had sessionid 0x13ad37f995c0000
[junit4:junit4]   2> 2124 T11 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33140/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@3909f81a
[junit4:junit4]   2> 2126 T19 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:33140
[junit4:junit4]   2> 2126 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2129 T19 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33140, initiating session
[junit4:junit4]   2> 2129 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:42943
[junit4:junit4]   2> 2130 T13 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:42943
[junit4:junit4]   2> 2133 T15 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad37f995c0001 with negotiated timeout 10000 for client /140.211.11.196:42943
[junit4:junit4]   2> 2133 T19 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:33140, sessionid = 0x13ad37f995c0001, negotiated timeout = 10000
[junit4:junit4]   2> 2135 T20 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3909f81a name:ZooKeeperConnection Watcher:127.0.0.1:33140/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2135 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2177 T11 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 2184 T11 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 2190 T11 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 2223 T11 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 2229 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2288 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2305 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 2307 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 2416 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2419 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2462 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 2464 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 2469 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 2471 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 2476 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 2478 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 2483 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 2485 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 2490 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 2492 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 2497 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 2499 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 2504 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 2506 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 2511 T17 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x13ad37f995c0001
[junit4:junit4]   2> 2513 T20 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 2513 T13 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:42943 which had sessionid 0x13ad37f995c0001
[junit4:junit4]   2> 2513 T11 oaz.ZooKeeper.close Session: 0x13ad37f995c0001 closed
[junit4:junit4]   2> 3178 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 3602 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:23671
[junit4:junit4]   2> 3832 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3833 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 3833 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4432 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 4436 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4437 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 4468 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 4473 T11 oasc.CoreContainer.<init> New CoreContainer 873441026
[junit4:junit4]   2> 4474 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4475 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4562 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4593 T11 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:33140/solr
[junit4:junit4]   2> 4631 T11 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33140/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@1fcebee
[junit4:junit4]   2> 4633 T35 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:33140
[junit4:junit4]   2> 4633 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4635 T35 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33140, initiating session
[junit4:junit4]   2> 4636 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:51841
[junit4:junit4]   2> 4637 T13 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:51841
[junit4:junit4]   2> 4639 T15 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad37f995c0002 with negotiated timeout 5000 for client /140.211.11.196:51841
[junit4:junit4]   2> 4639 T35 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:33140, sessionid = 0x13ad37f995c0002, negotiated timeout = 5000
[junit4:junit4]   2> 4640 T36 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fcebee name:ZooKeeperConnection Watcher:127.0.0.1:33140/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4641 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4658 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0002 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4668 T11 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 4671 T11 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:23671_solr
[junit4:junit4]   2> 4677 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0002 type:delete cxid:0x7 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:23671_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:23671_solr
[junit4:junit4]   2> 4678 T11 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:23671_solr
[junit4:junit4]   2> 4693 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 4694 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4695 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 4695 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4696 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4697 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4697 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4698 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4698 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4746 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 5206 T11 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 5229 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0002 type:delete cxid:0x15 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 5230 T11 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 5234 T11 oasc.Overseer.start Overseer (id=88615687674396674-127.0.0.1:23671_solr-n_0000000000) starting
[junit4:junit4]   2> 5235 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0002 type:create cxid:0x1a zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5251 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0002 type:create cxid:0x1b zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5252 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0002 type:create cxid:0x1c zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5263 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0002 type:create cxid:0x1d zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5265 T38 oasc.OverseerCollectionProcessor.run Process current queue of collection messages
[junit4:junit4]   2> 5265 T11 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 5269 T11 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 5293 T37 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 5347 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 5348 T11 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 5349 T11 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 5350 T11 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 5352 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 5354 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 5355 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 5619 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 5831 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 5961 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 6064 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 7765 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 7793 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 7797 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 8184 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8206 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8213 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8220 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8306 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/control/data/
[junit4:junit4]   2> 8315 T11 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ac91010
[junit4:junit4]   2> 8537 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/control/data/index/
[junit4:junit4]   2> 8539 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 8649 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/control/data/index forceNew:false
[junit4:junit4]   2> 9087 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@55958273; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 9088 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 9134 T11 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 10356 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 10357 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 10357 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 10368 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 10511 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 10526 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 10531 T11 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 10568 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 10593 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 10669 T11 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 10671 T11 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 10678 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 10679 T11 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 10680 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 10681 T11 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 10721 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 10722 T11 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10723 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10724 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10725 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10726 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10728 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10729 T11 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10755 T11 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 10781 T11 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 10965 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 11000 T11 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 11009 T11 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 11084 T11 oass.SolrIndexSearcher.<init> Opening Searcher@7377dfc2 main
[junit4:junit4]   2> 11212 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 11213 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 11214 T11 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 11387 T11 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 11807 T39 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7377dfc2 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 11826 T11 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11827 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0002 type:create cxid:0x77 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 11900 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 11901 T11 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:23671/solr shardId:control_shard
[junit4:junit4]   2> 11917 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 11940 T11 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/control_shard/election
[junit4:junit4]   2> 12016 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0002 type:delete cxid:0x86 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 12046 T11 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 12047 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0002 type:create cxid:0x87 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12327 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12328 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"control_shard",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:23671_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:23671/solr"}
[junit4:junit4]   2> 12344 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0002 type:create cxid:0x8e zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 12395 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (1)
[junit4:junit4]   2> 12558 T11 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 12558 T11 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 12559 T11 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:23671/solr/collection1/
[junit4:junit4]   2> 12560 T11 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 12560 T11 oasc.SyncStrategy.syncToMe http://127.0.0.1:23671/solr/collection1/ has no replicas
[junit4:junit4]   2> 12561 T11 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:23671/solr/collection1/
[junit4:junit4]   2> 12561 T11 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/control_shard
[junit4:junit4]   2> 12576 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0002 type:create cxid:0xac zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12903 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12917 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (1)
[junit4:junit4]   2> 12943 T11 oasc.ZkController.register We are http://127.0.0.1:23671/solr/collection1/ and leader is http://127.0.0.1:23671/solr/collection1/
[junit4:junit4]   2> 12946 T11 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:23671/solr
[junit4:junit4]   2> 12947 T11 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 12947 T11 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 12950 T11 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12951 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 12952 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 13000 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 13141 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 13144 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:54171
[junit4:junit4]   2> 13145 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 13145 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352168098346
[junit4:junit4]   2> 13146 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352168098346/'
[junit4:junit4]   2> 13178 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 13178 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 13179 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352168098346
[junit4:junit4]   2> 13179 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352168098346/solr.xml
[junit4:junit4]   2> 13180 T11 oasc.CoreContainer.<init> New CoreContainer 230897924
[junit4:junit4]   2> 13181 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352168098346/'
[junit4:junit4]   2> 13181 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352168098346/'
[junit4:junit4]   2> 13209 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 13224 T11 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:33140/solr
[junit4:junit4]   2> 13225 T11 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33140/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@70745955
[junit4:junit4]   2> 13226 T49 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:33140
[junit4:junit4]   2> 13226 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 13228 T49 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33140, initiating session
[junit4:junit4]   2> 13228 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:45948
[junit4:junit4]   2> 13229 T13 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:45948
[junit4:junit4]   2> 13231 T15 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad37f995c0003 with negotiated timeout 5000 for client /140.211.11.196:45948
[junit4:junit4]   2> 13231 T49 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:33140, sessionid = 0x13ad37f995c0003, negotiated timeout = 5000
[junit4:junit4]   2> 13232 T50 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@70745955 name:ZooKeeperConnection Watcher:127.0.0.1:33140/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 13232 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 13233 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0003 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13241 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0003 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13243 T11 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54171_solr
[junit4:junit4]   2> 13243 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0003 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:54171_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:54171_solr
[junit4:junit4]   2> 13245 T11 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:54171_solr
[junit4:junit4]   2> 13248 T36 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 13249 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 13249 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 13250 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 13250 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 13251 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 13251 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 13252 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 13252 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 13253 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 13253 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 13271 T11 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 13278 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352168098346/collection1
[junit4:junit4]   2> 13278 T11 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 13279 T11 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 13280 T11 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 13282 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352168098346/collection1/'
[junit4:junit4]   2> 13283 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty1-1352168098346/collection1/lib/README' to classloader
[junit4:junit4]   2> 13284 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty1-1352168098346/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 13394 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 13422 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13424 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"control_shard",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:23671_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:23671/solr"}
[junit4:junit4]   2> 13430 T50 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 13430 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 13489 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 13591 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 13662 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 14425 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 14439 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 14443 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 14470 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 14476 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 14482 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 14486 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 14489 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty1-1352168098346/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty1/
[junit4:junit4]   2> 14490 T11 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ac91010
[junit4:junit4]   2> 14491 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty1/index/
[junit4:junit4]   2> 14492 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 14493 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty1/index forceNew:false
[junit4:junit4]   2> 14499 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@49a52b21; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 14500 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 14501 T11 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 14505 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 14505 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 14506 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 14506 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 14507 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 14508 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 14509 T11 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 14510 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 14512 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 14513 T11 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 14515 T11 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 14516 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 14517 T11 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 14517 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 14518 T11 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 14520 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 14521 T11 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14523 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14524 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14525 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14527 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14528 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14530 T11 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14531 T11 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 14533 T11 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 14542 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 14546 T11 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 14548 T11 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 14553 T11 oass.SolrIndexSearcher.<init> Opening Searcher@66c0bc57 main
[junit4:junit4]   2> 14554 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 14554 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 14556 T11 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 14563 T11 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 14629 T55 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@66c0bc57 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 14640 T11 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14947 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14948 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:54171_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:54171/solr"}
[junit4:junit4]   2> 14949 T37 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 14960 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 14960 T50 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 15144 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 15144 T11 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:54171/solr shardId:shard1
[junit4:junit4]   2> 15146 T11 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 15170 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0003 type:delete cxid:0x43 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 15178 T11 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 15179 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0003 type:create cxid:0x44 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15188 T11 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 15189 T11 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 15189 T11 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:54171/solr/collection1/
[junit4:junit4]   2> 15190 T11 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 15190 T11 oasc.SyncStrategy.syncToMe http://127.0.0.1:54171/solr/collection1/ has no replicas
[junit4:junit4]   2> 15191 T11 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:54171/solr/collection1/
[junit4:junit4]   2> 15191 T11 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 15205 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0003 type:create cxid:0x4e zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15472 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15490 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 15491 T50 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 15513 T11 oasc.ZkController.register We are http://127.0.0.1:54171/solr/collection1/ and leader is http://127.0.0.1:54171/solr/collection1/
[junit4:junit4]   2> 15513 T11 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:54171/solr
[junit4:junit4]   2> 15514 T11 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 15514 T11 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15517 T11 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15518 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 15519 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 15519 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 15656 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 15659 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:54465
[junit4:junit4]   2> 15660 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 15661 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352168100860
[junit4:junit4]   2> 15661 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352168100860/'
[junit4:junit4]   2> 15694 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 15695 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 15696 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352168100860
[junit4:junit4]   2> 15696 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352168100860/solr.xml
[junit4:junit4]   2> 15697 T11 oasc.CoreContainer.<init> New CoreContainer 872227382
[junit4:junit4]   2> 15697 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352168100860/'
[junit4:junit4]   2> 15698 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352168100860/'
[junit4:junit4]   2> 15728 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 15745 T11 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:33140/solr
[junit4:junit4]   2> 15746 T11 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33140/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@6b3e3c75
[junit4:junit4]   2> 15747 T65 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:33140
[junit4:junit4]   2> 15747 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 15749 T65 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33140, initiating session
[junit4:junit4]   2> 15749 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:49378
[junit4:junit4]   2> 15750 T13 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:49378
[junit4:junit4]   2> 15751 T15 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad37f995c0004 with negotiated timeout 5000 for client /140.211.11.196:49378
[junit4:junit4]   2> 15751 T65 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:33140, sessionid = 0x13ad37f995c0004, negotiated timeout = 5000
[junit4:junit4]   2> 15752 T66 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6b3e3c75 name:ZooKeeperConnection Watcher:127.0.0.1:33140/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 15753 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 15754 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0004 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15755 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0004 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15757 T11 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54465_solr
[junit4:junit4]   2> 15758 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0004 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:54465_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:54465_solr
[junit4:junit4]   2> 15764 T11 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:54465_solr
[junit4:junit4]   2> 15768 T36 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 15768 T50 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 15769 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 15770 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 15770 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 15771 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 15771 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 15772 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 15772 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 15773 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 15773 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 15774 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 15791 T11 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 15799 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352168100860/collection1
[junit4:junit4]   2> 15800 T11 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 15801 T11 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 15802 T11 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 15804 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352168100860/collection1/'
[junit4:junit4]   2> 15806 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty2-1352168100860/collection1/lib/README' to classloader
[junit4:junit4]   2> 15808 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty2-1352168100860/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 15898 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 15961 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 16006 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16008 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:54171_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:54171/solr"}
[junit4:junit4]   2> 16021 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 16021 T50 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 16062 T66 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 16062 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 16103 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 16665 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 16679 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 16683 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 16731 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 16744 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 16755 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 16761 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 16767 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty2-1352168100860/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty2/
[junit4:junit4]   2> 16768 T11 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ac91010
[junit4:junit4]   2> 16772 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty2/index/
[junit4:junit4]   2> 16773 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 16775 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty2/index forceNew:false
[junit4:junit4]   2> 16790 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@255b252c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 16792 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 16794 T11 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 16801 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 16803 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 16804 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 16807 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 16809 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 16811 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 16813 T11 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 16816 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 16818 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 16821 T11 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 16824 T11 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 16826 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 16829 T11 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 16830 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 16832 T11 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 16835 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 16838 T11 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16841 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16843 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16846 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16849 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16852 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16855 T11 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16857 T11 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 16860 T11 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 16882 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 16892 T11 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 16895 T11 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 16905 T11 oass.SolrIndexSearcher.<init> Opening Searcher@327b9698 main
[junit4:junit4]   2> 16909 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 16910 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 16913 T11 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 16973 T11 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 17116 T71 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@327b9698 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 17146 T11 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 17534 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17536 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:54465_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:54465/solr"}
[junit4:junit4]   2> 17556 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 17557 T66 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 17557 T50 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 17650 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 17651 T11 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:54465/solr shardId:shard2
[junit4:junit4]   2> 17652 T11 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 17670 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0004 type:delete cxid:0x42 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 17671 T11 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 17672 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0004 type:create cxid:0x43 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 17681 T11 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 17682 T11 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 17682 T11 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:54465/solr/collection1/
[junit4:junit4]   2> 17683 T11 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 17683 T11 oasc.SyncStrategy.syncToMe http://127.0.0.1:54465/solr/collection1/ has no replicas
[junit4:junit4]   2> 17684 T11 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:54465/solr/collection1/
[junit4:junit4]   2> 17684 T11 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 17690 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0004 type:create cxid:0x4c zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 18068 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18095 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 18095 T50 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 18095 T66 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 18101 T11 oasc.ZkController.register We are http://127.0.0.1:54465/solr/collection1/ and leader is http://127.0.0.1:54465/solr/collection1/
[junit4:junit4]   2> 18101 T11 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:54465/solr
[junit4:junit4]   2> 18102 T11 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 18102 T11 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 18105 T11 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18107 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 18107 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 18108 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 18232 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 18234 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:37521
[junit4:junit4]   2> 18236 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 18236 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352168103448
[junit4:junit4]   2> 18237 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352168103448/'
[junit4:junit4]   2> 18266 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 18267 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 18268 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352168103448
[junit4:junit4]   2> 18268 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352168103448/solr.xml
[junit4:junit4]   2> 18268 T11 oasc.CoreContainer.<init> New CoreContainer 332661287
[junit4:junit4]   2> 18269 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352168103448/'
[junit4:junit4]   2> 18270 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352168103448/'
[junit4:junit4]   2> 18295 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 18309 T11 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:33140/solr
[junit4:junit4]   2> 18310 T11 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33140/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@4febd0d
[junit4:junit4]   2> 18311 T81 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:33140
[junit4:junit4]   2> 18311 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 18312 T81 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33140, initiating session
[junit4:junit4]   2> 18313 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:55601
[junit4:junit4]   2> 18313 T13 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:55601
[junit4:junit4]   2> 18316 T15 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad37f995c0005 with negotiated timeout 5000 for client /140.211.11.196:55601
[junit4:junit4]   2> 18316 T81 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:33140, sessionid = 0x13ad37f995c0005, negotiated timeout = 5000
[junit4:junit4]   2> 18316 T82 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4febd0d name:ZooKeeperConnection Watcher:127.0.0.1:33140/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 18317 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 18318 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0005 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 18320 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0005 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 18322 T11 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37521_solr
[junit4:junit4]   2> 18322 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0005 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:37521_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:37521_solr
[junit4:junit4]   2> 18330 T11 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37521_solr
[junit4:junit4]   2> 18334 T50 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 18334 T36 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 18334 T66 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 18337 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 18337 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 18338 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 18338 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 18338 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 18339 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 18339 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 18340 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 18340 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 18341 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 18357 T11 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 18363 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352168103448/collection1
[junit4:junit4]   2> 18364 T11 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 18365 T11 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 18365 T11 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 18367 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352168103448/collection1/'
[junit4:junit4]   2> 18368 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty3-1352168103448/collection1/lib/README' to classloader
[junit4:junit4]   2> 18369 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty3-1352168103448/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 18425 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 18485 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 18587 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 18603 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18605 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:54465_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:54465/solr"}
[junit4:junit4]   2> 18619 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 18619 T50 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 18619 T66 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 18619 T82 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 18621 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 19140 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 19153 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 19156 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 19183 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 19189 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 19193 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 19196 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 19199 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty3-1352168103448/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty3/
[junit4:junit4]   2> 19200 T11 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ac91010
[junit4:junit4]   2> 19201 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty3/index/
[junit4:junit4]   2> 19202 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 19203 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty3/index forceNew:false
[junit4:junit4]   2> 19245 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@53323184; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 19246 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 19248 T11 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 19252 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 19253 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 19254 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 19255 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 19258 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 19258 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 19259 T11 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 19261 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 19262 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 19264 T11 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 19265 T11 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 19267 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 19269 T11 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 19270 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 19271 T11 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 19274 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 19276 T11 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19277 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19279 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19280 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19290 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19291 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19292 T11 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19293 T11 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 19294 T11 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 19303 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 19307 T11 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 19308 T11 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 19313 T11 oass.SolrIndexSearcher.<init> Opening Searcher@53d4287 main
[junit4:junit4]   2> 19314 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 19314 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 19315 T11 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 19323 T11 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 19387 T87 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@53d4287 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 19394 T11 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 19629 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19631 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:37521_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:37521/solr"}
[junit4:junit4]   2> 19645 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 19645 T50 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 19645 T66 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 19645 T82 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 19898 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 19898 T11 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:37521/solr shardId:shard1
[junit4:junit4]   2> 19903 T11 oasc.ZkController.register We are http://127.0.0.1:37521/solr/collection1/ and leader is http://127.0.0.1:54171/solr/collection1/
[junit4:junit4]   2> 19904 T11 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:37521/solr
[junit4:junit4]   2> 19904 T11 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 19905 T11 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> 19912 T11 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19913 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 19913 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 19914 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C0 name=collection1 org.apache.solr.core.SolrCore@5636f88c url=http://127.0.0.1:37521/solr/collection1 node=127.0.0.1:37521_solr
[junit4:junit4]   2> 19921 T88 C0 P37521 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 19927 T88 C0 P37521 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 19927 T88 C0 P37521 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 19935 T88 C0 P37521 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:54171/solr/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 19948 T88 C0 P37521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 19976 T88 C0 P37521 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:37521/solr START replicas=[http://127.0.0.1:54171/solr/collection1/] nUpdates=100
[junit4:junit4]   2> 20000 T88 C0 P37521 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 20001 T88 C0 P37521 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 20002 T88 C0 P37521 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 20002 T88 C0 P37521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 20123 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 20127 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:32207
[junit4:junit4]   2> 20129 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 20130 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty4-1352168105256
[junit4:junit4]   2> 20131 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty4-1352168105256/'
[junit4:junit4]   2> 20151 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20152 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:37521_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:37521/solr"}
[junit4:junit4]   2> 20159 T50 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 20159 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 20159 T82 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 20159 T66 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 20222 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 20223 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 20224 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty4-1352168105256
[junit4:junit4]   2> 20225 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty4-1352168105256/solr.xml
[junit4:junit4]   2> 20226 T11 oasc.CoreContainer.<init> New CoreContainer 349497597
[junit4:junit4]   2> 20228 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty4-1352168105256/'
[junit4:junit4]   2> 20229 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty4-1352168105256/'
[junit4:junit4]   2> 20289 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 20314 T11 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:33140/solr
[junit4:junit4]   2> 20315 T11 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33140/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@84373ce
[junit4:junit4]   2> 20316 T99 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:33140
[junit4:junit4]   2> 20316 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 20319 T99 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33140, initiating session
[junit4:junit4]   2> 20319 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:49628
[junit4:junit4]   2> 20320 T13 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:49628
[junit4:junit4]   2> 20322 T15 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad37f995c0006 with negotiated timeout 5000 for client /140.211.11.196:49628
[junit4:junit4]   2> 20322 T99 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:33140, sessionid = 0x13ad37f995c0006, negotiated timeout = 5000
[junit4:junit4]   2> 20323 T100 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@84373ce name:ZooKeeperConnection Watcher:127.0.0.1:33140/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 20323 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 20324 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0006 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 20326 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0006 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 20328 T11 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32207_solr
[junit4:junit4]   2> 20329 T17 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ad37f995c0006 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:32207_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:32207_solr
[junit4:junit4]   2> 20330 T11 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:32207_solr
[junit4:junit4]   2> 20343 T50 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 20343 T82 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 20343 T66 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 20343 T36 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 20344 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 20345 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 20346 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 20346 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 20347 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 20348 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 20348 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 20349 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 20350 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 20351 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 20373 T11 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 20385 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty4-1352168105256/collection1
[junit4:junit4]   2> 20386 T11 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 20387 T11 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 20388 T11 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 20390 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty4-1352168105256/collection1/'
[junit4:junit4]   2> 20392 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty4-1352168105256/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 20394 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/solrtest-jetty4-1352168105256/collection1/lib/README' to classloader
[junit4:junit4]   2> 20531 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 20623 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 20725 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 20810 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> ASYNC  NEW_CORE C1 name=collection1 org.apache.solr.core.SolrCore@7edeaabc url=http://127.0.0.1:54171/solr/collection1 node=127.0.0.1:54171_solr
[junit4:junit4]   2> 21123 T52 C1 P54171 REQ /get {distrib=false&qt=/get&wt=javabin&version=2&getVersions=100} status=0 QTime=41 
[junit4:junit4]   2> 21824 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 21844 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 21849 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 21878 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 21890 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 21895 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 21908 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 21911 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-jetty4-1352168105256/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty4/
[junit4:junit4]   2> 21911 T11 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ac91010
[junit4:junit4]   2> 21913 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty4/index/
[junit4:junit4]   2> 21917 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 21918 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty4/index forceNew:false
[junit4:junit4]   2> 21970 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3bc02228; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 21971 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 21972 T11 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 21977 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 21977 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 21978 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 21978 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 21979 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 21980 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 21980 T11 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 21981 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 21982 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 21983 T11 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 21984 T11 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 21985 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 21986 T11 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 21986 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 21987 T11 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 21988 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 21989 T11 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 21990 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 21991 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 21992 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 21992 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 21993 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 21994 T11 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 21995 T11 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 21996 T11 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 22004 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 22008 T11 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 22010 T11 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 22013 T11 oass.SolrIndexSearcher.<init> Opening Searcher@9c89957 main
[junit4:junit4]   2> 22015 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 22015 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 22017 T11 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 22035 T11 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 22108 T104 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@9c89957 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 22117 T11 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 22179 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 22180 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:32207_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:32207/solr"}
[junit4:junit4]   2> 22193 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (5)
[junit4:junit4]   2> 22193 T66 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (5)
[junit4:junit4]   2> 22193 T100 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (5)
[junit4:junit4]   2> 22193 T50 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (5)
[junit4:junit4]   2> 22193 T82 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (5)
[junit4:junit4]   2> 22620 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 22620 T11 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:32207/solr shardId:shard2
[junit4:junit4]   2> 22624 T11 oasc.ZkController.register We are http://127.0.0.1:32207/solr/collection1/ and leader is http://127.0.0.1:54465/solr/collection1/
[junit4:junit4]   2> 22624 T11 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:32207/solr
[junit4:junit4]   2> 22625 T11 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 22625 T11 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C2 name=collection1 org.apache.solr.core.SolrCore@ce14ed3 url=http://127.0.0.1:32207/solr/collection1 node=127.0.0.1:32207_solr
[junit4:junit4]   2> 22626 T105 C2 P32207 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 22627 T11 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 22627 T105 C2 P32207 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 22627 T105 C2 P32207 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 22628 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 22629 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 22629 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 22631 T105 C2 P32207 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:54465/solr/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 22631 T105 C2 P32207 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:32207/solr START replicas=[http://127.0.0.1:54465/solr/collection1/] nUpdates=100
[junit4:junit4]   2> 22632 T105 C2 P32207 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 22632 T105 C2 P32207 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 22633 T105 C2 P32207 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 22633 T105 C2 P32207 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C3 name=collection1 org.apache.solr.core.SolrCore@7fe69211 url=http://127.0.0.1:54465/solr/collection1 node=127.0.0.1:54465_solr
[junit4:junit4]   2> 22649 T69 C3 P54465 REQ /get {distrib=false&qt=/get&wt=javabin&version=2&getVersions=100} status=0 QTime=1 
[junit4:junit4]   2> 22657 T11 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33140/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@4f77037a
[junit4:junit4]   2> 22658 T107 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:33140
[junit4:junit4]   2> 22658 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 22659 T107 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33140, initiating session
[junit4:junit4]   2> 22659 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:29328
[junit4:junit4]   2> 22660 T13 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:29328
[junit4:junit4]   2> 22661 T15 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ad37f995c0007 with negotiated timeout 10000 for client /140.211.11.196:29328
[junit4:junit4]   2> 22662 T107 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:33140, sessionid = 0x13ad37f995c0007, negotiated timeout = 10000
[junit4:junit4]   2> 22662 T108 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4f77037a name:ZooKeeperConnection Watcher:127.0.0.1:33140/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 22662 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 22663 T11 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 22667 T11 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 22668 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 22678 T11 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 22698 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 22700 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:32207_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:32207/solr"}
[junit4:junit4]   2> 22711 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (5)
[junit4:junit4]   2> 22711 T50 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (5)
[junit4:junit4]   2> 22711 T100 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (5)
[junit4:junit4]   2> 22711 T82 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (5)
[junit4:junit4]   2> 22711 T66 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (5)
[junit4:junit4]   2> 22711 T108 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (5)
[junit4:junit4]   2> ASYNC  NEW_CORE C4 name=collection1 org.apache.solr.core.SolrCore@2bc1e78c url=http://127.0.0.1:23671/solr/collection1 node=127.0.0.1:23671_solr
[junit4:junit4]   2> 22810 T32 C4 P23671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16
[junit4:junit4]   2>  C0_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:37521_solr, base_url=http://127.0.0.1:37521/solr}
[junit4:junit4]   2> 23214 T88 C0 P37521 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 23214 T88 C0 P37521 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 23215 T88 C0 P37521 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:54171/solr/collection1/. core=collection1
[junit4:junit4]   2> 23215 T88 C0 P37521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 23260 T53 C1 P54171 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 23268 T53 C1 P54171 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-1352168085567/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@49a52b21; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 23269 T53 C1 P54171 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 23271 T53 C1 P54171 oass.SolrIndexSearcher.<init> Opening Searcher@58a386c4 realtime
[junit4:junit4]   2> 23271 T53 C1 P54171 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 23272 T53 C1 P54171 /update {waitSearcher=true&openSearcher=false&commit=true&commit_end_point=true&wt=javabin&softCommit=false&version=2} {commit=} 0 32
[junit4:junit4]   2> 23338 T88 C0 P37521 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 23338 T88 C0 P37521 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4]   2> 23366 T88 C0 P37521 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 2

[...truncated too long message...]

erver.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  21) Thread[id=112, name=TEST-FullSolrCloudDistribCmdsTest.testDistribSearch-seed#[3AA65C7CB87FC8FA]-SendThread(localhost:38567), state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4:junit4]   2> 	        at sun.nio.ch.IOUtil.drain(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.updateSelectedKeys(KQueueSelectorImpl.java:167)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:108)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1131)
[junit4:junit4]   2> 	  22) Thread[id=60, name=qtp2054141666-60 Selector1, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.Timeout.expired(Timeout.java:97)
[junit4:junit4]   2> 	        at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:686)
[junit4:junit4]   2> 	        at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:285)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  23) Thread[id=30, name=qtp504163721-30 Acceptor1 SelectChannelConnector@0.0.0.0:24570, state=BLOCKED, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  24) Thread[id=106, name=searcherExecutor-41-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  25) Thread[id=66, name=qtp2054141666-66 Acceptor3 SelectChannelConnector@0.0.0.0:42265, state=BLOCKED, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  26) Thread[id=62, name=qtp2054141666-62 Selector3, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueArrayWrapper.interrupt(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueArrayWrapper.interrupt(KQueueArrayWrapper.java:164)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.wakeup(KQueueSelectorImpl.java:245)
[junit4:junit4]   2> 	        at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:210)
[junit4:junit4]   2> 	        at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:216)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:102)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4]   2> 	        at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:564)
[junit4:junit4]   2> 	        at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:285)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  27) Thread[id=95, name=qtp1627197352-95 Selector3, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:159)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.selectNow(SelectorImpl.java:106)
[junit4:junit4]   2> 	        at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:531)
[junit4:junit4]   2> 	        at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:285)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  28) Thread[id=99, name=qtp1627197352-99 Acceptor3 SelectChannelConnector@0.0.0.0:65270, state=BLOCKED, group=TGRP-FullSolrCloudDistribCmdsTest]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Appending, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=fi, timezone=Asia/Irkutsk
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=36,free=83267720,total=135266304
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestDocSet, FullSolrCloudDistribCmdsTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=FullSolrCloudDistribCmdsTest -Dtests.seed=3AA65C7CB87FC8FA -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=fi -Dtests.timezone=Asia/Irkutsk -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] ERROR   0.00s J1 | FullSolrCloudDistribCmdsTest (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([3AA65C7CB87FC8FA]:0)
[junit4:junit4] Completed on J1 in 7223.97s, 1 test, 2 errors <<< FAILURES!

[...truncated 184 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:335: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/module-build.xml:61: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:1163: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:827: There were test failures: 233 suites, 965 tests, 3 suite-level errors, 1 error, 1 failure, 878 ignored

Total time: 138 minutes 1 second
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 611 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/611/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.TestDistributedSearch

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([43BDC8EDB8C9CCE8]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.update.SolrCmdDistributorTest

Error Message:
20 threads leaked from SUITE scope at org.apache.solr.update.SolrCmdDistributorTest:     1) Thread[id=61, name=qtp1984873408-61 Acceptor1 SelectChannelConnector@0.0.0.0:54029, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=48, name=qtp1173231242-48 Acceptor2 SelectChannelConnector@0.0.0.0:21937, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=33, name=qtp1271263678-33 Acceptor1 SelectChannelConnector@0.0.0.0:31121, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=60, name=qtp1984873408-60 Acceptor0 SelectChannelConnector@0.0.0.0:54029, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=74, name=qtp1189333092-74 Acceptor0 SelectChannelConnector@0.0.0.0:40308, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=76, name=qtp1189333092-76 Acceptor2 SelectChannelConnector@0.0.0.0:40308, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=77, name=qtp1189333092-77 Acceptor3 SelectChannelConnector@0.0.0.0:40308, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=75, name=qtp1189333092-75 Acceptor1 SelectChannelConnector@0.0.0.0:40308, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=46, name=qtp1173231242-46 Acceptor0 SelectChannelConnector@0.0.0.0:21937, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=32, name=qtp1271263678-32 Acceptor0 SelectChannelConnector@0.0.0.0:31121, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=15, name=qtp191968531-15 Acceptor1 SelectChannelConnector@0.0.0.0:55383, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=19, name=qtp191968531-19 Acceptor0 SelectChannelConnector@0.0.0.0:55383, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=47, name=qtp1173231242-47 Acceptor1 SelectChannelConnector@0.0.0.0:21937, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=62, name=qtp1984873408-62 Acceptor2 SelectChannelConnector@0.0.0.0:54029, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=20, name=qtp191968531-20 Acceptor2 SelectChannelConnector@0.0.0.0:55383, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=21, name=qtp191968531-21 Acceptor3 SelectChannelConnector@0.0.0.0:55383, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=35, name=qtp1271263678-35 Acceptor3 SelectChannelConnector@0.0.0.0:31121, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=49, name=qtp1173231242-49 Acceptor3 SelectChannelConnector@0.0.0.0:21937, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=34, name=qtp1271263678-34 Acceptor2 SelectChannelConnector@0.0.0.0:31121, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=63, name=qtp1984873408-63 Acceptor3 SelectChannelConnector@0.0.0.0:54029, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 20 threads leaked from SUITE scope at org.apache.solr.update.SolrCmdDistributorTest: 
   1) Thread[id=61, name=qtp1984873408-61 Acceptor1 SelectChannelConnector@0.0.0.0:54029, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=48, name=qtp1173231242-48 Acceptor2 SelectChannelConnector@0.0.0.0:21937, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=33, name=qtp1271263678-33 Acceptor1 SelectChannelConnector@0.0.0.0:31121, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=60, name=qtp1984873408-60 Acceptor0 SelectChannelConnector@0.0.0.0:54029, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=74, name=qtp1189333092-74 Acceptor0 SelectChannelConnector@0.0.0.0:40308, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=76, name=qtp1189333092-76 Acceptor2 SelectChannelConnector@0.0.0.0:40308, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=77, name=qtp1189333092-77 Acceptor3 SelectChannelConnector@0.0.0.0:40308, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=75, name=qtp1189333092-75 Acceptor1 SelectChannelConnector@0.0.0.0:40308, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=46, name=qtp1173231242-46 Acceptor0 SelectChannelConnector@0.0.0.0:21937, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=32, name=qtp1271263678-32 Acceptor0 SelectChannelConnector@0.0.0.0:31121, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=15, name=qtp191968531-15 Acceptor1 SelectChannelConnector@0.0.0.0:55383, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=19, name=qtp191968531-19 Acceptor0 SelectChannelConnector@0.0.0.0:55383, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=47, name=qtp1173231242-47 Acceptor1 SelectChannelConnector@0.0.0.0:21937, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=62, name=qtp1984873408-62 Acceptor2 SelectChannelConnector@0.0.0.0:54029, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=20, name=qtp191968531-20 Acceptor2 SelectChannelConnector@0.0.0.0:55383, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=21, name=qtp191968531-21 Acceptor3 SelectChannelConnector@0.0.0.0:55383, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=35, name=qtp1271263678-35 Acceptor3 SelectChannelConnector@0.0.0.0:31121, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=49, name=qtp1173231242-49 Acceptor3 SelectChannelConnector@0.0.0.0:21937, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=34, name=qtp1271263678-34 Acceptor2 SelectChannelConnector@0.0.0.0:31121, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=63, name=qtp1984873408-63 Acceptor3 SelectChannelConnector@0.0.0.0:54029, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([43BDC8EDB8C9CCE8]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.update.SolrCmdDistributorTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=61, name=qtp1984873408-61 Acceptor1 SelectChannelConnector@0.0.0.0:54029, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=33, name=qtp1271263678-33 Acceptor1 SelectChannelConnector@0.0.0.0:31121, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=48, name=qtp1173231242-48 Acceptor2 SelectChannelConnector@0.0.0.0:21937, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=60, name=qtp1984873408-60 Acceptor0 SelectChannelConnector@0.0.0.0:54029, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=74, name=qtp1189333092-74 Acceptor0 SelectChannelConnector@0.0.0.0:40308, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=76, name=qtp1189333092-76 Acceptor2 SelectChannelConnector@0.0.0.0:40308, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=75, name=qtp1189333092-75 Acceptor1 SelectChannelConnector@0.0.0.0:40308, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=77, name=qtp1189333092-77 Acceptor3 SelectChannelConnector@0.0.0.0:40308, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=32, name=qtp1271263678-32 Acceptor0 SelectChannelConnector@0.0.0.0:31121, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=46, name=qtp1173231242-46 Acceptor0 SelectChannelConnector@0.0.0.0:21937, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=15, name=qtp191968531-15 Acceptor1 SelectChannelConnector@0.0.0.0:55383, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=19, name=qtp191968531-19 Acceptor0 SelectChannelConnector@0.0.0.0:55383, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=47, name=qtp1173231242-47 Acceptor1 SelectChannelConnector@0.0.0.0:21937, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=62, name=qtp1984873408-62 Acceptor2 SelectChannelConnector@0.0.0.0:54029, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=20, name=qtp191968531-20 Acceptor2 SelectChannelConnector@0.0.0.0:55383, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=21, name=qtp191968531-21 Acceptor3 SelectChannelConnector@0.0.0.0:55383, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=35, name=qtp1271263678-35 Acceptor3 SelectChannelConnector@0.0.0.0:31121, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=49, name=qtp1173231242-49 Acceptor3 SelectChannelConnector@0.0.0.0:21937, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=34, name=qtp1271263678-34 Acceptor2 SelectChannelConnector@0.0.0.0:31121, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=63, name=qtp1984873408-63 Acceptor3 SelectChannelConnector@0.0.0.0:54029, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=61, name=qtp1984873408-61 Acceptor1 SelectChannelConnector@0.0.0.0:54029, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=33, name=qtp1271263678-33 Acceptor1 SelectChannelConnector@0.0.0.0:31121, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=48, name=qtp1173231242-48 Acceptor2 SelectChannelConnector@0.0.0.0:21937, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=60, name=qtp1984873408-60 Acceptor0 SelectChannelConnector@0.0.0.0:54029, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=74, name=qtp1189333092-74 Acceptor0 SelectChannelConnector@0.0.0.0:40308, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=76, name=qtp1189333092-76 Acceptor2 SelectChannelConnector@0.0.0.0:40308, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=75, name=qtp1189333092-75 Acceptor1 SelectChannelConnector@0.0.0.0:40308, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=77, name=qtp1189333092-77 Acceptor3 SelectChannelConnector@0.0.0.0:40308, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=32, name=qtp1271263678-32 Acceptor0 SelectChannelConnector@0.0.0.0:31121, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=46, name=qtp1173231242-46 Acceptor0 SelectChannelConnector@0.0.0.0:21937, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=15, name=qtp191968531-15 Acceptor1 SelectChannelConnector@0.0.0.0:55383, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=19, name=qtp191968531-19 Acceptor0 SelectChannelConnector@0.0.0.0:55383, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=47, name=qtp1173231242-47 Acceptor1 SelectChannelConnector@0.0.0.0:21937, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=62, name=qtp1984873408-62 Acceptor2 SelectChannelConnector@0.0.0.0:54029, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=20, name=qtp191968531-20 Acceptor2 SelectChannelConnector@0.0.0.0:55383, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=21, name=qtp191968531-21 Acceptor3 SelectChannelConnector@0.0.0.0:55383, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=35, name=qtp1271263678-35 Acceptor3 SelectChannelConnector@0.0.0.0:31121, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=49, name=qtp1173231242-49 Acceptor3 SelectChannelConnector@0.0.0.0:21937, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=34, name=qtp1271263678-34 Acceptor2 SelectChannelConnector@0.0.0.0:31121, state=BLOCKED, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=63, name=qtp1984873408-63 Acceptor3 SelectChannelConnector@0.0.0.0:54029, state=RUNNABLE, group=TGRP-SolrCmdDistributorTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([43BDC8EDB8C9CCE8]:0)


FAILED:  org.apache.solr.TestDistributedSearch.testDistribSearch

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([43BDC8EDB8C9CCE8]:0)




Build Log:
[...truncated 8298 lines...]
[junit4:junit4] Suite: org.apache.solr.update.SolrCmdDistributorTest
[junit4:junit4]   2> 276 T11 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> 914 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 1326 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:55383
[junit4:junit4]   2> 1529 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1530 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 1531 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2378 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 2383 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2384 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 2425 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 2430 T11 oasc.CoreContainer.<init> New CoreContainer 1019014919
[junit4:junit4]   2> 2432 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2432 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2532 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 2628 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 2630 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 2632 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 2633 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 2718 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 2898 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 2925 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 3015 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 4792 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 4829 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 4834 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5218 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5240 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5246 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5265 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5362 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/control/data/
[junit4:junit4]   2> 5363 T11 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 5431 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/control/data/index/
[junit4:junit4]   2> 5433 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1352114291911/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5553 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/control/data/index forceNew:false
[junit4:junit4]   2> 5976 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@2a16f5e lockFactory=org.apache.lucene.store.NativeFSLockFactory@296dfc2d; maxCacheMB=0.572265625 maxMergeSizeMB=0.55078125)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5977 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 7109 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 7135 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 7177 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 7184 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 7371 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 7481 T11 oass.SolrIndexSearcher.<init> Opening Searcher@834f574 main
[junit4:junit4]   2> 7482 T11 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 7605 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 7606 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 7738 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 7739 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 7740 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 7740 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 7741 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 7741 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 7742 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 7743 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 7743 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 7796 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 8429 T26 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@834f574 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 8432 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 8433 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 8434 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 8512 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 8538 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 8540 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:31121
[junit4:junit4]   2> 8542 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 8542 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 8543 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 8590 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 8591 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 8592 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 8592 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 8593 T11 oasc.CoreContainer.<init> New CoreContainer 156027432
[junit4:junit4]   2> 8594 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 8594 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 8630 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 8664 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 8665 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 8667 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 8668 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 8720 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 8779 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 8780 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 8832 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 9464 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 9481 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 9485 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 9508 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9514 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9519 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9522 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9525 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard0/data/
[junit4:junit4]   2> 9526 T11 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 9527 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard0/data/index/
[junit4:junit4]   2> 9528 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard0/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 9529 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard0/data/index forceNew:false
[junit4:junit4]   2> 9533 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@64712baa lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fc0dff4),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 9534 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 9536 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 9536 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 9537 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 9538 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 9553 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 9558 T11 oass.SolrIndexSearcher.<init> Opening Searcher@1373ef79 main
[junit4:junit4]   2> 9558 T11 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 9560 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 9560 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 9561 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 9562 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 9562 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 9563 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 9563 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 9564 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 9564 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 9565 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 9565 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 9566 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 9580 T40 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1373ef79 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 9579 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 9581 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 9581 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 9582 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9594 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 9597 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:21937
[junit4:junit4]   2> 9597 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 9598 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 9599 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 9642 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 9642 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 9643 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 9643 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 9644 T11 oasc.CoreContainer.<init> New CoreContainer 235102498
[junit4:junit4]   2> 9645 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 9645 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 9679 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 9700 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 9701 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 9702 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 9704 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 9750 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 9815 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 9816 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 9862 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 10428 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 10442 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 10446 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 10467 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10472 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10477 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10479 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10482 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard1/data/
[junit4:junit4]   2> 10483 T11 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 10484 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard1/data/index/
[junit4:junit4]   2> 10485 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard1/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 10486 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard1/data/index forceNew:false
[junit4:junit4]   2> 10489 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1416e4cd lockFactory=org.apache.lucene.store.NativeFSLockFactory@2f66531d),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 10490 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 10491 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 10492 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 10492 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 10493 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 10503 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 10513 T11 oass.SolrIndexSearcher.<init> Opening Searcher@6063eda7 main
[junit4:junit4]   2> 10513 T11 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 10514 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 10515 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 10516 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 10516 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 10517 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 10517 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 10518 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 10518 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 10519 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 10519 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 10520 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 10520 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 10534 T54 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6063eda7 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 10534 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 10535 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 10536 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 10537 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 10548 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 10551 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:54029
[junit4:junit4]   2> 10551 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 10552 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 10552 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 10584 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 10585 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 10586 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 10586 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 10587 T11 oasc.CoreContainer.<init> New CoreContainer 402434553
[junit4:junit4]   2> 10587 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 10588 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 10615 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 10634 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 10635 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 10637 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 10638 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 10675 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 10720 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 10721 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 10761 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 11356 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 11370 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 11373 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 11394 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 11399 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 11404 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 11406 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 11409 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard2/data/
[junit4:junit4]   2> 11409 T11 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 11410 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard2/data/index/
[junit4:junit4]   2> 11411 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard2/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 11412 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard2/data/index forceNew:false
[junit4:junit4]   2> 11416 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1eea7695 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2e367152),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 11416 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 11418 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 11418 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 11419 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 11420 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 11431 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 11437 T11 oass.SolrIndexSearcher.<init> Opening Searcher@5096a235 main
[junit4:junit4]   2> 11438 T11 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 11439 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 11440 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 11440 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 11441 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 11442 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 11442 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 11443 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 11443 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 11444 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 11444 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 11445 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 11445 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 11459 T68 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5096a235 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 11459 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 11460 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 11461 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 11461 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 11473 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 11476 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:40308
[junit4:junit4]   2> 11476 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 11477 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 11477 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 11511 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 11512 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 11512 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 11513 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 11513 T11 oasc.CoreContainer.<init> New CoreContainer 82190788
[junit4:junit4]   2> 11514 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 11515 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 11544 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 11562 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 11563 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 11564 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 11565 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 11612 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 11661 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-tlog.xml
[junit4:junit4]   2> 11662 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 11696 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 12292 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 12309 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 12313 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 12333 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 12338 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 12343 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 12345 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 12347 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard3/data/
[junit4:junit4]   2> 12347 T11 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4]   2> 12349 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard3/data/index/
[junit4:junit4]   2> 12350 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard3/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 12350 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./org.apache.solr.update.SolrCmdDistributorTest-1352114291911/shard3/data/index forceNew:false
[junit4:junit4]   2> 12354 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@9a99335 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6390f540),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 12354 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 12356 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 12356 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 12357 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 12357 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 12367 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 12372 T11 oass.SolrIndexSearcher.<init> Opening Searcher@753d011 main
[junit4:junit4]   2> 12373 T11 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 12374 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 12375 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 12375 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 12376 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 12376 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 12377 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 12377 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 12378 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 12378 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 12379 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 12379 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 12380 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 12395 T82 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@753d011 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 12395 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 12396 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 12397 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 12397 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C0 name=collection1 org.apache.solr.core.SolrCore@6e21b5ab
[junit4:junit4]   2> 14154 T24 C0 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@2a16f5e lockFactory=org.apache.lucene.store.NativeFSLockFactory@296dfc2d; maxCacheMB=0.572265625 maxMergeSizeMB=0.55078125)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 14156 T24 C0 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 14203 T24 C0 oass.SolrIndexSearcher.<init> Opening Searcher@31394b77 realtime
[junit4:junit4]   2> 14204 T24 C0 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 14211 T24 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1417794610009210880)} 0 620
[junit4:junit4]   2> ASYNC  NEW_CORE C1 name=collection1 org.apache.solr.core.SolrCore@27e1d8a2
[junit4:junit4]   2> 14405 T38 C1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@64712baa lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fc0dff4),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 14406 T38 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 14408 T38 C1 oass.SolrIndexSearcher.<init> Opening Searcher@7c004572 realtime
[junit4:junit4]   2> 14408 T38 C1 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 14409 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1417794610562859008)} 0 7
[junit4:junit4]   2> ASYNC  NEW_CORE C2 name=collection1 org.apache.solr.core.SolrCore@18c18838
[junit4:junit4]   2> 14422 T53 C2 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1416e4cd lockFactory=org.apache.lucene.store.NativeFSLockFactory@2f66531d),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 14423 T53 C2 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 14425 T53 C2 oass.SolrIndexSearcher.<init> Opening Searcher@2ad2ee23 realtime
[junit4:junit4]   2> 14426 T53 C2 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 14427 T53 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1417794610578587648)} 0 9
[junit4:junit4]   2> ASYNC  NEW_CORE C3 name=collection1 org.apache.solr.core.SolrCore@701eec73
[junit4:junit4]   2> 14446 T65 C3 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1eea7695 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2e367152),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 14447 T65 C3 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 14449 T65 C3 oass.SolrIndexSearcher.<init> Opening Searcher@5141185c realtime
[junit4:junit4]   2> 14450 T65 C3 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 14450 T65 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1417794610600607744)} 0 11
[junit4:junit4]   2> ASYNC  NEW_CORE C4 name=collection1 org.apache.solr.core.SolrCore@429302e2
[junit4:junit4]   2> 14461 T80 C4 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@9a99335 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6390f540),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 14461 T80 C4 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 14463 T80 C4 oass.SolrIndexSearcher.<init> Opening Searcher@24ed587c realtime
[junit4:junit4]   2> 14463 T80 C4 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 14464 T80 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1417794610620530688)} 0 6
[junit4:junit4]   2> 14486 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16
[junit4:junit4]   2> 14961 T22 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1 (1417794610794594304)]} 0 348
[junit4:junit4]   2> 14969 T11 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:55383/solr/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4]   2> 14993 T22 C0 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15119 T22 C0 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@2a16f5e lockFactory=org.apache.lucene.store.NativeFSLockFactory@296dfc2d; maxCacheMB=0.572265625 maxMergeSizeMB=0.55078125)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@2a16f5e lockFactory=org.apache.lucene.store.NativeFSLockFactory@296dfc2d; maxCacheMB=0.572265625 maxMergeSizeMB=0.55078125)),segFN=segments_2,generation=2,filenames=[_0_Lucene40_0.tim, _0.fnm, _0_Lucene40_0.frq, _0_nrm.cfe, segments_2, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene40_0.prx, _0.fdt, _0_Lucene40_0.tip]
[junit4:junit4]   2> 15119 T22 C0 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 15251 T22 C0 oass.SolrIndexSearcher.<init> Opening Searcher@53670068 main
[junit4:junit4]   2> 15252 T22 C0 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 15256 T22 C0 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15269 T26 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@53670068 main{StandardDirectoryReader(segments_2:3:nrt _0(4.1):C1)}
[junit4:junit4]   2> 15270 T22 C0 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&commit_end_point=true&expungeDeletes=false&wt=javabin&softCommit=false&version=2} {commit=} 0 277
[junit4:junit4]   2> 15545 T23 C0 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=1 status=0 QTime=243 
[junit4:junit4]   2> 15602 T22 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&commit_end_point=true&version=2} {add=[2 (1417794611815907328), 3 (1417794611819053056)]} 0 5
[junit4:junit4]   2> 15603 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&commit_end_point=true&version=2} {add=[2 (1417794611814858752), 3 (1417794611816955904), 4 (1417794611819053056)]} 0 6
[junit4:junit4]   2> 15604 T11 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:55383/solr/, StdNode: http://127.0.0.1:31121/solr/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4]   2> 15608 T24 C0 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15608 T37 C1 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15613 T37 C1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@64712baa lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fc0dff4),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@64712baa lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fc0dff4),segFN=segments_2,generation=2,filenames=[_0_Lucene40_0.tim, _0.fnm, _0_Lucene40_0.frq, _0_nrm.cfe, segments_2, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene40_0.prx, _0.fdt, _0_Lucene40_0.tip]
[junit4:junit4]   2> 15614 T37 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 15616 T37 C1 oass.SolrIndexSearcher.<init> Opening Searcher@6516b1d9 main
[junit4:junit4]   2> 15616 T37 C1 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 15618 T40 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6516b1d9 main{StandardDirectoryReader(segments_2:3:nrt _0(4.1):C3)}
[junit4:junit4]   2> 15618 T37 C1 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15618 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&commit_end_point=true&expungeDeletes=false&wt=javabin&softCommit=false&version=2} {commit=} 0 10
[junit4:junit4]   2> 15658 T24 C0 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@2a16f5e lockFactory=org.apache.lucene.store.NativeFSLockFactory@296dfc2d; maxCacheMB=0.572265625 maxMergeSizeMB=0.55078125)),segFN=segments_2,generation=2,filenames=[_0_Lucene40_0.tim, _0.fnm, _0_Lucene40_0.frq, _0_nrm.cfe, segments_2, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene40_0.prx, _0.fdt, _0_Lucene40_0.tip]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@2a16f5e lockFactory=org.apache.lucene.store.NativeFSLockFactory@296dfc2d; maxCacheMB=0.572265625 maxMergeSizeMB=0.55078125)),segFN=segments_3,generation=3,filenames=[_1_Lucene40_0.tim, _1_nrm.cfs, _0_Lucene40_0.tim, _0.fnm, _1_Lucene40_0.frq, _0_nrm.cfs, _0_Lucene40_0.prx, _1.fnm, _1_Lucene40_0.prx, _1_Lucene40_0.tip, _1_nrm.cfe, _0_Lucene40_0.tip, _1.fdx, _0_Lucene40_0.frq, _1.fdt, _1.si, _0_nrm.cfe, _0.fdx, _0.si, segments_3, _0.fdt]
[junit4:junit4]   2> 15659 T24 C0 oasc.SolrDeletionPolicy.updateCommits newest commit = 3
[junit4:junit4]   2> 15668 T24 C0 oass.SolrIndexSearcher.<init> Opening Searcher@586d9632 main
[junit4:junit4]   2> 15668 T24 C0 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 15669 T24 C0 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15669 T26 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@586d9632 main{StandardDirectoryReader(segments_3:5:nrt _0(4.1):C1 _1(4.1):C2)}
[junit4:junit4]   2> 15670 T24 C0 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&commit_end_point=true&expungeDeletes=false&wt=javabin&softCommit=false&version=2} {commit=} 0 62
[junit4:junit4]   2> 15674 T23 C0 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=3 status=0 QTime=1 
[junit4:junit4]   2> 15683 T38 C1 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=3 status=0 QTime=1 
[junit4:junit4]   2> 15691 T22 C0 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&commit_end_point=true&version=2} {delete=[2 (-1417794611912376320)]} 0 1
[junit4:junit4]   2> 15691 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&commit_end_point=true&version=2} {delete=[2 (-1417794611912376320)]} 0 1
[junit4:junit4]   2> 15693 T11 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:55383/solr/, StdNode: http://127.0.0.1:31121/solr/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4]   2> 15696 T37 C1 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15696 T24 C0 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15710 T37 C1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@64712baa lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fc0dff4),segFN=segments_2,generation=2,filenames=[_0_Lucene40_0.tim, _0.fnm, _0_Lucene40_0.frq, _0_nrm.cfe, segments_2, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene40_0.prx, _0.fdt, _0_Lucene40_0.tip]
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@64712baa lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fc0dff4),segFN=segments_3,generation=3,filenames=[_0_Lucene40_0.tim, _0.fnm, _0_Lucene40_0.frq, _0_nrm.cfe, _0_1.del, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene40_0.prx, _0.fdt, segments_3, _0_Lucene40_0.tip]
[junit4:junit4]   2> 15711 T37 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 3
[junit4:junit4]   2> 15711 T24 C0 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@2a16f5e lockFactory=org.apache.lucene.store.NativeFSLockFactory@296dfc2d; maxCacheMB=0.572265625 maxMergeSizeMB=0.55078125)),segFN=segments_3,generation=3,filenames=[_1_Lucene40_0.tim, _1_nrm.cfs, _0_Lucene40_0.tim, _0.fnm, _1_Lucene40_0.frq, _0_nrm.cfs, _0_Lucene40_0.prx, _1.fnm, _1_Lucene40_0.prx, _1_Lucene40_0.tip, _1_nrm.cfe, _0_Lucene40_0.tip, _1.fdx, _0_Lucene40_0.frq, _1.fdt, _1.si, _0_nrm.cfe, _0.fdx, _0.si, segments_3, _0.fdt]
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@2a16f5e lockFactory=org.apache.lucene.store.NativeFSLockFactory@296dfc2d; maxCacheMB=0.572265625 maxMergeSizeMB=0.55078125)),segFN=segments_4,generation=4,filenames=[_1_Lucene40_0.tim, _1_nrm.cfs, _0_Lucene40_0.tim, _0.fnm, _1_Lucene40_0.frq, _0_nrm.cfs, _1_1.del, _0_Lucene40_0.prx, _1.fnm, _1_Lucene40_0.prx, _1_Lucene40_0.tip, _1_nrm.cfe, _0_Lucene40_0.tip, _1.fdx, _0_Lucene40_0.frq, _1.fdt, _1.si, _0_nrm.cfe, _0.fdx, _0.si, _0.fdt, segments_4]
[junit4:junit4]   2> 15712 T37 C1 oass.SolrIndexSearcher.<init> Opening Searcher@14b58fc0 main
[junit4:junit4]   2> 15712 T24 C0 oasc.SolrDeletionPolicy.updateCommits newest commit = 4
[junit4:junit4]   2> 15712 T37 C1 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 15713 T40 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@14b58fc0 main{StandardDirectoryReader(segments_3:4:nrt _0(4.1):C3/1)}
[junit4:junit4]   2> 15713 T37 C1 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15714 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&commit_end_point=true&expungeDeletes=false&wt=javabin&softCommit=false&version=2} {commit=} 0 18
[junit4:junit4]   2> 15716 T24 C0 oass.SolrIndexSearcher.<init> Opening Searcher@15ec1723 main
[junit4:junit4]   2> 15717 T24 C0 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 15718 T26 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@15ec1723 main{StandardDirectoryReader(segments_4:6:nrt _0(4.1):C1 _1(4.1):C2/1)}
[junit4:junit4]   2> 15718 T24 C0 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15719 T24 C0 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&commit_end_point=true&expungeDeletes=false&wt=javabin&softCommit=false&version=2} {commit=} 0 23
[junit4:junit4]   2> 15723 T23 C0 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=2 status=0 QTime=1 
[junit4:junit4]   2> 15728 T38 C1 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=*:*&version=2} hits=2 status=0 QTime=1 
[junit4:junit4]   2> 15732 T39 C1 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15974 T39 C1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@64712baa lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fc0dff4),segFN=segments_3,generation=3,filenames=[_0_Lucene40_0.tim, _0.fnm, _0_Lucene40_0.frq, _0_nrm.cfe, _0_1.del, _0.fdx, _0_nrm.cfs, _0.si, _0_Lucene40_0.prx, _0.fdt, segments_3, _0_Lucene40_0.tip]
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@64712baa lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fc0dff4),segFN=segments_4,generation=4,filenames=[_1_Lucene40_0.tim, _1.fdx, _1_nrm.cfs, _1.fdt, _1.si, _1_Lucene40_0.frq, _1.fnm, _1_Lucene40_0.prx, _1_Lucene40_0.tip, _1_nrm.cfe, segments_4]
[junit4:junit4]   2> 15975 T39 C1 oasc.SolrDeletionPolicy.updateCommits newest commit = 4
[junit4:junit4]   2> 15979 T39 C1 oass.SolrIndexSearcher.<init> Opening Searcher@75a3a7f7 main
[junit4:junit4]   2> 15979 T39 C1 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 15980 T39 C1 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15981 T40 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@75a3a7f7 main{StandardDirectoryReader(segments_4:6:nrt _1(4.1):C2)}
[junit4:junit4]   2> 15982 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={optimize=true&maxSegments=1&waitSearcher=true&wt=javabin&version=2} {optimize=} 0 250
[junit4:junit4]   2> 15987 T52 C2 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15989 T52 C2 oass.SolrIndexSearcher.<init> Opening Searcher@4a0a4516 main
[junit4:junit4]   2> 15989 T52 C2 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 15990 T54 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4a0a4516 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4]   2> 15990 T52 C2 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15991 T52 C2 UPDATE [collection1] webapp=/solr path=/update params={optimize=true&maxSegments=1&waitSearcher=true&wt=javabin&version=2} {optimize=} 0 4
[junit4:junit4]   2> 15995 T67 C3 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15996 T67 C3 oass.SolrIndexSearcher.<init> Opening Searcher@4038e358 main
[junit4:junit4]   2> 15997 T67 C3 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 15998 T68 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4038e358 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4]   2> 15998 T67 C3 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15998 T67 C3 UPDATE [collection1] webapp=/solr path=/update params={optimize=true&maxSegments=1&waitSearcher=true&wt=javabin&version=2} {optimize=} 0 3
[junit4:junit4]   2> 16002 T78 C4 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 16004 T78 C4 oass.SolrIndexSearcher.<init> Opening Searcher@401bb340 main
[junit4:junit4]   2> 16004 T78 C4 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 16005 T82 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@401bb340 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4]   2> 16005 T78 C4 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 16006 T78 C4 UPDATE [collection1] webapp=/solr path=/update params={optimize=true&maxSegments=1&waitSearcher=true&wt=javabin&version=2} {optimize=} 0 4
[junit4:junit4]   2> 16058 T81 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[5 (1417794612268892160), 7 (1417794612276232192), 9 (1417794612278329344), 10 (1417794612281475073), 11 (1417794612283572225), 12 (1417794612285669376), 14 (1417794612287766528), 17 (1417794612289863680), 18 (1417794612293009409), 20 (1417794612295106560)]} 0 27
[junit4:junit4]   2> 16072 T78 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[21 (1417794612273086464), 22 (1417794612276232193), 23 (1417794612281475072), 24 (1417794612283572224), 25 (1417794612284620800), 26 (1417794612285669377), 30 (1417794612286717952), 31 (1417794612287766529), 32 (1417794612289863681), 34 (1417794612304543744)]} 0 38
[junit4:junit4]   2> 16084 T65 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[5 (1417794612270989312), 7 (1417794612276232193), 9 (1417794612281475072), 11 (1417794612291960833), 12 (1417794612297203712), 16 (1417794612300349440), 19 (1417794612303495168), 21 (1417794612307689473), 23 (1417794612312932353), 25 (1417794612317126656)]} 0 53
[junit4:junit4]   2> 16089 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[49 (1417794612289863680), 50 (1417794612295106561), 51 (1417794612296155137), 53 (1417794612298252288), 54 (1417794612300349440), 56 (1417794612301398017), 58 (1417794612302446593), 59 (1417794612307689473), 60 (1417794612313980928), 61 (1417794612328660993)]} 0 39
[junit4:junit4]   2> 16090 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[35 (1417794612274135040), 36 (1417794612291960832), 37 (1417794612293009408), 38 (1417794612296155136), 40 (1417794612299300864), 41 (1417794612301398016), 42 (1417794612307689472), 44 (1417794612312932352), 46 (1417794612315029504), 48 (1417794612317126656)]} 0 55
[junit4:junit4]   2> 16091 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[6 (1417794612272037888), 13 (1417794612279377920), 15 (1417794612291960833), 22 (1417794612295106560), 27 (1417794612297203712), 28 (1417794612300349441), 29 (1417794612302446592), 30 (1417794612309786624), 31 (1417794612328660992), 32 (1417794612330758144)]} 0 59
[junit4:junit4]   2> 16099 T52 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[5 (1417794612269940736), 7 (1417794612275183616), 10 (1417794612288815105), 12 (1417794612293009408), 13 (1417794612299300864), 17 (1417794612302446592), 19 (1417794612307689472), 20 (1417794612311883776), 22 (1417794612332855298), 23 (1417794612338098177)]} 0 69
[junit4:junit4]   2> 16101 T66 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[27 (1417794612276232192), 28 (1417794612291960832), 29 (1417794612293009408), 33 (1417794612298252288), 34 (1417794612305592320), 37 (1417794612309786624), 39 (1417794612315029505), 42 (1417794612318175232), 43 (1417794612326563840), 44 (1417794612328660992)]} 0 65
[junit4:junit4]   2> 16116 T67 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[47 (1417794612289863680), 50 (1417794612312932352), 52 (1417794612323418112), 53 (1417794612327612416), 54 (1417794612329709568), 58 (1417794612343341056), 61 (1417794612347535360), 63 (1417794612350681089), 64 (1417794612353826816), 65 (1417794612355923968)]} 0 66
[junit4:junit4]   2> 16116 T106 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[64 (1417794612310835200), 66 (1417794612327612416), 68 (1417794612333903872), 74 (1417794612342292480), 79 (1417794612345438208), 81 (1417794612346486784), 82 (1417794612347535361), 85 (1417794612350681088), 86 (1417794612352778240), 88 (1417794612355923968)]} 0 46
[junit4:junit4]   2> 16126 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[183 (1417794612347535360), 184 (1417794612349632512), 185 (1417794612351729664), 189 (1417794612354875392), 190 (1417794612358021120), 191 (1417794612360118272), 192 (1417794612362215424), 193 (1417794612363264000), 194 (1417794612365361152), 195 (1417794612367458304)]} 0 22
[junit4:junit4]   2> 16150 T100 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[64 (1417794612298252288), 67 (1417794612308738048), 68 (1417794612328660992), 69 (1417794612332855296), 72 (1417794612334952450), 73 (1417794612341243904), 74 (1417794612344389632), 76 (1417794612345438208), 78 (1417794612347535360), 79 (1417794612389478400)]} 0 93
[junit4:junit4]   2> 16154 T65 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[108 (1417794612339146752), 113 (1417794612346486784), 129 (1417794612348583936), 131 (1417794612350681088), 133 (1417794612351729664), 140 (1417794612354875392), 142 (1417794612358021120), 143 (1417794612360118272), 144 (1417794612362215424), 147 (1417794612394721280)]} 0 58
[junit4:junit4]   2> 16159 T53 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[27 (1417794612273086464), 29 (1417794612288815104), 31 (1417794612291960832), 33 (1417794612297203712), 36 (1417794612300349440), 37 (1417794612303495168), 39 (1417794612309786624), 41 (1417794612323418113), 42 (1417794612332855296), 46 (1417794612336001024)]} 0 125
[junit4:junit4]   2> 16161 T80 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[35 (1417794612277280768), 41 (1417794612288815104), 43 (1417794612293009408), 45 (1417794612306640896), 47 (1417794612322369537), 52 (1417794612329709568), 55 (1417794612330758145), 56 (1417794612334952448), 60 (1417794612392624129), 61 (1417794612397867008)]} 0 123
[junit4:junit4]   2> 16172 T102 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[66 (1417794612307689472), 67 (1417794612315029504), 68 (1417794612332855296), 69 (1417794612345438208), 70 (1417794612349632512), 72 (1417794612352778240), 74 (1417794612395769856), 76 (1417794612399964161), 77 (1417794612410449920), 78 (1417794612413595648)]} 0 106
[junit4:junit4]   2> 16173 T112 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[101 (1417794612323418112), 102 (1417794612330758144), 103 (1417794612333903873), 108 (1417794612387381248), 109 (1417794612407304192), 110 (1417794612409401345), 111 (1417794612412547072), 112 (1417794612413595648), 114 (1417794612415692800), 115 (1417794612416741377)]} 0 91
[junit4:junit4]   2> 16176 T51 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[50 (1417794612289863680), 52 (1417794612308738048), 53 (1417794612315029504), 56 (1417794612317126656), 57 (1417794612323418114), 59 (1417794612328660992), 60 (1417794612338098176), 64 (1417794612341243904), 65 (1417794612416741378), 66 (1417794612418838528)]} 0 126
[junit4:junit4]   2> 16186 T108 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[90 (1417794612316078080), 91 (1417794612330758145), 92 (1417794612332855296), 93 (1417794612387381248), 95 (1417794612391575552), 96 (1417794612396818434), 97 (1417794612401012736), 102 (1417794612409401344), 103 (1417794612413595648), 107 (1417794612426178562)]} 0 111
[junit4:junit4]   2> 16189 T105 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[85 (1417794612313980928), 86 (1417794612325515264), 92 (1417794612328660993), 94 (1417794612394721281), 96 (1417794612397867008), 101 (1417794612399964160), 102 (1417794612406255616), 103 (1417794612409401344), 106 (1417794612426178560), 107 (1417794612431421440)]} 0 117
[junit4:junit4]   2> 16190 T81 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[81 (1417794612310835200), 84 (1417794612315029504), 86 (1417794612317126656), 87 (1417794612322369536), 89 (1417794612324466688), 91 (1417794612329709569), 92 (1417794612334952449), 93 (1417794612392624128), 95 (1417794612397867009), 100 (1417794612423032833)]} 0 120
[junit4:junit4]   2> 16198 T114 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[108 (1417794612331806720), 111 (1417794612393672704), 112 (1417794612398915584), 115 (1417794612407304192), 116 (1417794612410449920), 119 (1417794612423032833), 122 (1417794612426178563), 123 (1417794612428275714), 124 (1417794612437712897), 125 (1417794612439810048)]} 0 114
[junit4:junit4]   2> 16200 T125 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[133 (1417794612342292480), 135 (1417794612391575553), 138 (1417794612395769856), 140 (1417794612399964160), 141 (1417794612408352768), 146 (1417794612424081408), 147 (1417794612427227137), 148 (1417794612431421440), 149 (1417794612434567169), 153 (1417794612438761473)]} 0 101
[junit4:junit4]   2> 16202 T103 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[85 (1417794612312932353), 92 (1417794612316078080), 93 (1417794612323418112), 94 (1417794612326563840), 95 (1417794612334952448), 98 (1417794612339146753), 100 (1417794612404158464), 102 (1417794612416741379), 103 (1417794612419887104), 106 (1417794612421984256)]} 0 130
[junit4:junit4]   2> 16204 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[131 (1417794612339146752), 134 (1417794612389478400), 135 (1417794612392624129), 136 (1417794612396818433), 138 (1417794612398915585), 139 (1417794612405207040), 142 (1417794612424081408), 143 (1417794612428275713), 144 (1417794612430372864), 145 (1417794612448198656)]} 0 108
[junit4:junit4]   2> 16209 T122 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[138 (1417794612339146752), 144 (1417794612416741376), 145 (1417794612424081408), 146 (1417794612427227136), 147 (1417794612429324288), 148 (1417794612431421441), 150 (1417794612433518593), 153 (1417794612435615745), 156 (1417794612452392960), 157 (1417794612454490112)]} 0 113
[junit4:junit4]   2> 16216 T67 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[192 (1417794612399964162), 194 (1417794612405207041), 197 (1417794612412547072), 203 (1417794612415692800), 204 (1417794612421984256), 207 (1417794612427227137), 208 (1417794612431421441), 211 (1417794612435615744), 212 (1417794612438761473), 213 (1417794612455538691)]} 0 61
[junit4:junit4]   2> 16231 T128 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[154 (1417794612391575552), 155 (1417794612409401344), 162 (1417794612426178560), 163 (1417794612429324288), 164 (1417794612432470018), 165 (1417794612437712896), 166 (1417794612440858624), 167 (1417794612449247234), 174 (1417794612453441536), 175 (1417794612460781569)]} 0 84
[junit4:junit4]   2> 16238 T101 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[68 (1417794612298252288), 69 (1417794612312932352), 71 (1417794612333903872), 72 (1417794612341243905), 73 (1417794612416741377), 74 (1417794612428275712), 77 (1417794612431421440), 78 (1417794612433518592), 80 (1417794612435615744), 83 (1417794612453441536)]} 0 181
[junit4:junit4]   2> 16239 T126 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[165 (1417794612396818432), 167 (1417794612416741376), 170 (1417794612419887105), 171 (1417794612421984256), 172 (1417794612433518592), 173 (1417794612437712896), 174 (1417794612448198657), 175 (1417794612452392961), 178 (1417794612462878720), 182 (1417794612480704512)]} 0 88
[junit4:junit4]   2> 16241 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[274 (1417794612412547072), 275 (1417794612415692800), 276 (1417794612419887104), 277 (1417794612423032832), 278 (1417794612435615744), 280 (1417794612438761473), 281 (1417794612453441536), 283 (1417794612457635841), 284 (1417794612479655936), 285 (1417794612486995968)]} 0 73
[junit4:junit4]   2> 16246 T146 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[268 (1417794612421984257), 269 (1417794612427227136), 270 (1417794612432470016), 271 (1417794612434567169), 272 (1417794612437712896), 274 (1417794612439810048), 275 (1417794612446101504), 276 (1417794612452392960), 277 (1417794612454490113), 279 (1417794612490141696)]} 0 70
[junit4:junit4]   2> 16254 T100 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[218 (1417794612405207042), 219 (1417794612412547073), 222 (1417794612414644224), 223 (1417794612416741376), 225 (1417794612418838528), 228 (1417794612420935680), 229 (1417794612423032832), 230 (1417794612445052928), 232 (1417794612460781568), 233 (1417794612497481728)]} 0 95
[junit4:junit4]   2> 16264 T136 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[235 (1417794612432470016), 236 (1417794612449247232), 237 (1417794612453441538), 239 (1417794612455538690), 240 (1417794612461830144), 241 (1417794612480704513), 242 (1417794612486995968), 244 (1417794612503773184), 246 (1417794612507967488), 247 (1417794612511113217)]} 0 78
[junit4:junit4]   2> 16265 T66 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[174 (1417794612390526976), 176 (1417794612404158464), 178 (1417794612425129984), 184 (1417794612444004352), 185 (1417794612452392961), 186 (1417794612458684416), 187 (1417794612495384576), 188 (1417794612499578880), 189 (1417794612507967491), 191 (1417794612511113216)]} 0 119
[junit4:junit4]   2> 16267 T129 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[149 (1417794612344389632), 150 (1417794612392624128), 152 (1417794612416741376), 153 (1417794612419887105), 154 (1417794612444004353), 156 (1417794612447150080), 160 (1417794612450295809), 164 (1417794612455538689), 169 (1417794612481753088), 171 (1417794612514258944)]} 0 166
[junit4:junit4]   2> 16267 T80 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[271 (1417794612432470017), 274 (1417794612436664320), 276 (1417794612439810049), 277 (1417794612447150081), 278 (1417794612451344384), 282 (1417794612455538689), 283 (1417794612457635840), 291 (1417794612477558784), 292 (1417794612509016065), 293 (1417794612515307520)]} 0 81
[junit4:junit4]   2> 16267 T51 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[220 (1417794612433518594), 222 (1417794612439810048), 227 (1417794612455538689), 228 (1417794612457635841), 233 (1417794612495384576), 234 (1417794612502724608), 235 (1417794612504821761), 239 (1417794612507967488), 242 (1417794612510064640), 244 (1417794612512161792)]} 0 80
[junit4:junit4]   2> 16292 T117 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[107 (1417794612332855297), 111 (1417794612387381248), 112 (1417794612450295808), 114 (1417794612453441537), 118 (1417794612457635840), 119 (1417794612478607362), 124 (1417794612483850241), 128 (1417794612489093121), 133 (1417794612503773184), 137 (1417794612506918913)]} 0 201
[junit4:junit4]   2> 16293 T106 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[229 (1417794612410449921), 231 (1417794612426178561), 232 (1417794612444004354), 235 (1417794612448198658), 238 (1417794612452392960), 240 (1417794612456587264), 242 (1417794612486995969), 246 (1417794612495384577), 247 (1417794612498530304), 250 (1417794612513210368)]} 0 128
[junit4:junit4]   2> 16288 T103 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[334 (1417794612488044544), 336 (1417794612506918912), 338 (1417794612522647555), 339 (1417794612524744706), 340 (1417794612526841856), 341 (1417794612528939011), 344 (1417794612531036162), 346 (1417794612534181889), 347 (1417794612536279040), 350 (1417794612537327616)]} 0 50
[junit4:junit4]   2> 16286 T147 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[248 (1417794612451344385), 250 (1417794612455538688), 254 (1417794612478607361), 255 (1417794612483850240), 259 (1417794612505870337), 260 (1417794612522647552), 261 (1417794612525793283), 262 (1417794612528939009), 263 (1417794612532084737), 265 (1417794612535230464)]} 0 88
[junit4:junit4]   2> 16295 T108 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[252 (1417794612434567168), 253 (1417794612436664320), 254 (1417794612438761472), 255 (1417794612440858624), 258 (1417794612449247232), 263 (1417794612453441538), 265 (1417794612458684416), 266 (1417794612479655937), 269 (1417794612483850240), 270 (1417794612489093121)]} 0 108
[junit4:junit4]   2> 16285 T151 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[309 (1417794612478607360), 311 (1417794612486995968), 312 (1417794612493287424), 314 (1417794612497481728), 318 (1417794612505870338), 323 (1417794612522647553), 324 (1417794612525793280), 326 (1417794612528939008), 327 (1417794612532084738), 333 (1417794612534181888)]} 0 56
[junit4:junit4]   2> 16285 T145 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[266 (1417794612420935681), 267 (1417794612460781568), 268 (1417794612481753089), 269 (1417794612489093120), 270 (1417794612497481729), 274 (1417794612499578882), 276 (1417794612524744705), 278 (1417794612527890434), 282 (1417794612531036160), 285 (1417794612534181891)]} 0 110
[junit4:junit4]   2> 16283 T122 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[351 (1417794612496433152), 354 (1417794612505870336), 359 (1417794612515307521), 361 (1417794612519501824), 362 (1417794612522647554), 363 (1417794612524744704), 365 (1417794612525793282), 368 (1417794612527890433), 371 (1417794612529987584), 372 (1417794612532084739)]} 0 36
[junit4:junit4]   2> 16276 T52 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[178 (1417794612395769856), 180 (1417794612451344384), 181 (1417794612455538690), 182 (1417794612463927297), 183 (1417794612480704512), 184 (1417794612482801664), 188 (1417794612489093122), 189 (1417794612499578881), 190 (1417794612521598976), 192 (1417794612523696129)]} 0 126
[junit4:junit4]   2> 16303 T120 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[198 (1417794612397867008), 201 (1417794612418838528), 206 (1417794612426178560), 208 (1417794612428275712), 209 (1417794612444004353), 210 (1417794612453441537), 213 (1417794612457635840), 224 (1417794612478607360), 227 (1417794612481753088), 228 (1417794612495384576)]} 0 150
[junit4:junit4]   2> 16275 T112 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[252 (1417794612425129984), 253 (1417794612427227136), 255 (1417794612428275712), 258 (1417794612449247233), 260 (1417794612480704512), 261 (1417794612510064641), 262 (1417794612513210368), 266 (1417794612519501824), 268 (1417794612521598976), 269 (1417794612523696129)]} 0 96
[junit4:junit4]   2> 16273 T133 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[197 (1417794612405207041), 198 (1417794612435615744), 202 (1417794612439810048), 203 (1417794612447150080), 204 (1417794612451344385), 205 (1417794612455538688), 206 (1417794612458684416), 208 (1417794612493287425), 214 (1417794612517404672), 216 (1417794612519501826)]} 0 114
[junit4:junit4]   2> 16270 T53 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[194 (1417794612409401344), 199 (1417794612420935680), 200 (1417794612448198656), 201 (1417794612457635842), 203 (1417794612477558784), 204 (1417794612481753088), 208 (1417794612494336000), 212 (1417794612499578880), 213 (1417794612515307520), 215 (1417794612517404672)]} 0 106
[junit4:junit4]   2> 16304 T81 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[294 (1417794612462878720), 295 (1417794612494336000), 296 (1417794612499578880), 297 (1417794612506918914), 300 (1417794612509016066), 302 (1417794612545716224), 303 (1417794612548861952), 304 (1417794612550959105), 305 (1417794612553056256), 308 (1417794612555153408)]} 0 90
[junit4:junit4]   2> 16294 T101 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[287 (1417794612495384577), 290 (1417794612520550401), 291 (1417794612525793281), 298 (1417794612528939010), 299 (1417794612531036161), 301 (1417794612534181890), 304 (1417794612536279041), 305 (1417794612539424768), 306 (1417794612541521920), 307 (1417794612543619072)]} 0 48
[junit4:junit4]   2> 16293 T100 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[371 (1417794612523696128), 372 (1417794612525793280), 373 (1417794612527890432), 375 (1417794612529987584), 377 (1417794612532084736), 379 (1417794612534181888), 380 (1417794612536279040), 381 (1417794612537327616), 382 (1417794612539424768), 383 (1417794612540473344)]} 0 21
[junit4:junit4]   2> 16293 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[147 (1417794612339146753), 148 (1417794612392624128), 149 (1417794612396818435), 152 (1417794612399964160), 155 (1417794612406255616), 156 (1417794612444004352), 157 (1417794612460781568), 161 (1417794612490141696), 162 (1417794612505870336), 164 (1417794612516356096)]} 0 197
[junit4:junit4]   2> 16310 T102 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[252 (1417794612419887104), 253 (1417794612425129985), 255 (1417794612429324288), 257 (1417794612433518592), 260 (1417794612438761472), 261 (1417794612448198656), 262 (1417794612455538688), 264 (1417794612457635840), 265 (1417794612507967492), 266 (1417794612550959105)]} 0 136
[junit4:junit4]   2> 16311 T78 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[116 (1417794612333903872), 118 (1417794612339146752), 120 (1417794612387381249), 124 (1417794612392624130), 125 (1417794612408352769), 128 (1417794612421984256), 129 (1417794612446101504), 130 (1417794612479655936), 131 (1417794612511113218), 132 (1417794612549910528)]} 0 219
[junit4:junit4]   2> 16292 T132 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[179 (1417794612405207040), 182 (1417794612434567168), 183 (1417794612438761472), 184 (1417794612440858625), 185 (1417794612448198656), 187 (1417794612453441537), 191 (1417794612456587264), 194 (1417794612490141696), 195 (1417794612509016064), 196 (1417794612512161793)]} 0 133
[junit4:junit4]   2> 16291 T127 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[158 (1417794612390526976), 161 (1417794612463927296), 165 (1417794612504821760), 166 (1417794612520550400), 167 (1417794612523696128), 171 (1417794612527890432), 172 (1417794612532084736), 173 (1417794612535230465), 174 (1417794612537327617), 177 (1417794612540473344)]} 0 145
[junit4:junit4]   2> 16312 T65 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[214 (1417794612407304192), 216 (1417794612426178561), 218 (1417794612430372864), 224 (1417794612434567168), 225 (1417794612454490112), 226 (1417794612458684417), 228 (1417794612479655937), 229 (1417794612502724608), 230 (1417794612550959106), 233 (1417794612560396289)]} 0 150
[junit4:junit4]   2> 16305 T150 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[305 (1417794612478607360), 307 (1417794612486995968), 309 (1417794612502724609), 312 (1417794612507967489), 313 (1417794612510064641), 314 (1417794612515307522), 315 (1417794612517404672), 316 (1417794612543619073), 317 (1417794612546764801), 318 (1417794612555153408)]} 0 75
[junit4:junit4]   2> 16319 T139 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[234 (1417794612405207040), 235 (1417794612450295808), 238 (1417794612455538690), 242 (1417794612489093120), 244 (1417794612515307520), 246 (1417794612544667649), 247 (1417794612550959104), 248 (1417794612558299136), 250 (1417794612562493440), 251 (1417794612567736320)]} 0 159
[junit4:junit4]   2> 16320 T148 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[357 (1417794612497481728), 358 (1417794612503773184), 363 (1417794612513210368), 364 (1417794612517404673), 365 (1417794612542570496), 366 (1417794612546764800), 367 (1417794612549910528), 368 (1417794612553056256), 372 (1417794612556201985), 373 (1417794612566687744)]} 0 71
[junit4:junit4]   2> 16326 T51 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[390 (1417794612545716225), 391 (1417794612548861952), 392 (1417794612550959104), 393 (1417794612552007681), 394 (1417794612554104832), 395 (1417794612556201984), 396 (1417794612558299136), 397 (1417794612560396289), 398 (1417794612562493440), 399 (1417794612563542016)]} 0 33
[junit4:junit4]   2> 16325 T67 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[319 (1417794612479655936), 321 (1417794612481753089), 328 (1417794612507967488), 330 (1417794612510064640), 332 (1417794612515307521), 333 (1417794612544667648), 336 (1417794612552007681), 337 (1417794612557250560), 338 (1417794612571930625), 339 (1417794612575076353)]} 0 95
[junit4:junit4]   2> 16325 T158 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[351 (1417794612493287424), 354 (1417794612496433152), 357 (1417794612498530304), 359 (1417794612510064640), 360 (1417794612512161792), 362 (1417794612514258944), 363 (1417794612519501825), 365 (1417794612552007681), 366 (1417794612554104832), 369 (1417794612575076353)]} 0 81
[junit4:junit4]   2> 16319 T154 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[287 (1417794612482801665), 288 (1417794612489093120), 289 (1417794612497481728), 290 (1417794612503773186), 294 (1417794612506918912), 297 (1417794612546764800), 300 (1417794612550959106), 303 (1417794612556201985), 307 (1417794612559347712), 308 (1417794612566687744)]} 0 86
[junit4:junit4]   2> 16329 T149 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[285 (1417794612482801664), 288 (1417794612489093121), 291 (1417794612507967490), 296 (1417794612512161792), 297 (1417794612543619074), 299 (1417794612545716225), 301 (1417794612554104832), 302 (1417794612560396288), 303 (1417794612564590592), 304 (1417794612569833472)]} 0 95
[junit4:junit4]   2> 16329 T153 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[311 (1417794612488044544), 312 (1417794612493287424), 313 (1417794612503773184), 314 (1417794612550959104), 316 (1417794612553056257), 317 (1417794612556201986), 319 (1417794612564590593), 320 (1417794612567736321), 321 (1417794612575076353), 324 (1417794612578222080)]} 0 91
[junit4:junit4]   2> 16341 T146 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[374 (1417794612552007680), 377 (1417794612556201984), 379 (1417794612558299137), 382 (1417794612561444864), 386 (1417794612563542016), 390 (1417794612575076352), 391 (1417794612577173504), 397 (1417794612583464960), 399 (1417794612586610688), 402 (1417794612589756416)]} 0 42
[junit4:junit4]   2> 16343 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[383 (1417794612522647552), 385 (1417794612550959105), 386 (1417794612553056256), 387 (1417794612555153408), 388 (1417794612557250560), 390 (1417794612564590592), 391 (1417794612566687746), 395 (1417794612568784896), 396 (1417794612577173504), 398 (1417794612579270657)]} 0 72
[junit4:junit4]   2> 16346 T105 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[340 (1417794612493287424), 342 (1417794612497481729), 343 (1417794612543619072), 345 (1417794612545716224), 346 (1417794612553056257), 347 (1417794612569833473), 348 (1417794612586610689), 352 (1417794612589756417), 353 (1417794612594999297), 355 (1417794612597096450)]} 0 102
[junit4:junit4]   2> 16348 T112 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[404 (1417794612558299136), 405 (1417794612560396288), 406 (1417794612562493440), 410 (1417794612563542018), 413 (1417794612575076352), 414 (1417794612577173504), 415 (1417794612586610690), 419 (1417794612589756417), 420 (1417794612591853568), 421 (1417794612600242179)]} 0 43
[junit4:junit4]   2> 16349 T114 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[325 (1417794612479655938), 327 (1417794612482801664), 330 (1417794612488044545), 331 (1417794612503773185), 332 (1417794612518453248), 334 (1417794612560396288), 336 (1417794612576124928), 338 (1417794612592902145), 339 (1417794612597096448), 343 (1417794612601290753)]} 0 118
[junit4:junit4]   2> 16351 T128 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[326 (1417794612481753088), 327 (1417794612490141697), 331 (1417794612506918913), 332 (1417794612545716225), 335 (1417794612549910529), 340 (1417794612561444864), 341 (1417794612564590592), 346 (1417794612568784896), 349 (1417794612587659264), 350 (1417794612599193601)]} 0 119
[junit4:junit4]   2> 16353 T161 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[373 (1417794612545716224), 376 (1417794612548861953), 380 (1417794612552007680), 382 (1417794612554104833), 383 (1417794612557250560), 384 (1417794612560396288), 385 (1417794612569833472), 386 (1417794612582416385), 387 (1417794612592902145), 389 (1417794612603387904)]} 0 60
[junit4:junit4]   2> 16354 T129 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[420 (1417794612571930624), 421 (1417794612574027776), 422 (1417794612576124929), 423 (1417794612578222080), 427 (1417794612584513537), 428 (1417794612593950720), 431 (1417794612596047872), 433 (1417794612600242176), 435 (1417794612602339328), 437 (1417794612605485056)]} 0 35
[junit4:junit4]   2> 16354 T126 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[344 (1417794612512161792), 345 (1417794612515307520), 346 (1417794612543619072), 347 (1417794612548861952), 348 (1417794612554104832), 353 (1417794612562493440), 355 (1417794612570882048), 356 (1417794612593950721), 358 (1417794612597096449), 360 (1417794612605485057)]} 0 93
[junit4:junit4]   2> 16357 T51 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[481 (1417794612581367808), 482 (1417794612583464961), 483 (1417794612585562112), 484 (1417794612589756418), 485 (1417794612591853568), 486 (1417794612593950720), 488 (1417794612596047872), 490 (1417794612597096448), 491 (1417794612603387905), 492 (1417794612609679361)]} 0 29
[junit4:junit4]   2> 16360 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[363 (1417794612516356097), 367 (1417794612542570496), 370 (1417794612544667648), 372 (1417794612556201984), 375 (1417794612569833472), 376 (1417794612586610688), 377 (1417794612598145026), 379 (1417794612603387904), 380 (1417794612606533632), 381 (1417794612611776512)]} 0 94
[junit4:junit4]   2> 16365 T66 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[403 (1417794612559347712), 405 (1417794612584513538), 408 (1417794612587659264), 409 (1417794612590804993), 410 (1417794612593950722), 411 (1417794612597096449), 415 (1417794612609679361), 416 (1417794612612825088), 418 (1417794612614922241), 419 (1417794612618067968)]} 0 59
[junit4:junit4]   2> 16372 T65 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[461 (1417794612584513536), 462 (1417794612586610690), 463 (1417794612590804992), 464 (1417794612593950721), 465 (1417794612597096448), 470 (1417794612612825089), 476 (1417794612615970817), 477 (1417794612621213696), 481 (1417794612623310848), 483 (1417794612625408002)]} 0 42
[junit4:junit4]   2> 16372 T150 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[485 (1417794612594999296), 486 (1417794612608630784), 489 (1417794612609679362), 494 (1417794612611776513), 496 (1417794612613873665), 497 (1417794612615970816), 502 (1417794612620165120), 504 (1417794612621213697), 505 (1417794612623310849), 509 (1417794612625408001)]} 0 32
[junit4:junit4]   2> 16372 T125 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[309 (1417794612506918912), 310 (1417794612511113216), 311 (1417794612552007680), 312 (1417794612563542017), 318 (1417794612582416384), 319 (1417794612599193602), 321 (1417794612610727936), 322 (1417794612617019392), 323 (1417794612621213698), 325 (1417794612624359424)]} 0 151
[junit4:junit4]   2> 16373 T160 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[416 (1417794612565639168), 417 (1417794612582416384), 418 (1417794612584513537), 419 (1417794612589756417), 420 (1417794612590804992), 421 (1417794612601290752), 423 (1417794612609679360), 426 (1417794612621213696), 429 (1417794612624359424), 433 (1417794612626456576)]} 0 61
[junit4:junit4]   2> 16382 T166 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[482 (1417794612592902144), 483 (1417794612602339329), 484 (1417794612606533632), 485 (1417794612611776512), 489 (1417794612619116545), 491 (1417794612621213696), 492 (1417794612623310849), 494 (1417794612625408001), 495 (1417794612628553728), 496 (1417794612635893761)]} 0 43
[junit4:junit4]   2> 16382 T80 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[442 (1417794612581367808), 445 (1417794612582416385), 446 (1417794612586610688), 447 (1417794612600242177), 448 (1417794612602339328), 451 (1417794612609679360), 453 (1417794612614922240), 456 (1417794612619116549), 457 (1417794612633796609), 458 (1417794612635893760)]} 0 55
[junit4:junit4]   2> 16382 T148 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[439 (1417794612576124928), 444 (1417794612604436480), 445 (1417794612607582208), 447 (1417794612611776512), 450 (1417794612613873664), 452 (1417794612622262272), 454 (1417794612625408000), 455 (1417794612627505152), 457 (1417794612633796608), 459 (1417794612635893760)]} 0 60
[junit4:junit4]   2> 16383 T53 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[401 (1417794612559347712), 403 (1417794612564590592), 404 (1417794612571930624), 406 (1417794612580319232), 407 (1417794612584513536), 408 (1417794612592902144), 409 (1417794612598145024), 410 (1417794612609679363), 413 (1417794612613873664), 414 (1417794612618067968)]} 0 76
[junit4:junit4]   2> 16384 T122 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[494 (1417794612602339328), 496 (1417794612604436481), 497 (1417794612609679364), 498 (1417794612612825088), 499 (1417794612615970816), 500 (1417794612620165121), 501 (1417794612623310849), 504 (1417794612625408001), 505 (1417794612628553728), 506 (1417794612636942336)]} 0 39
[junit4:junit4]   2> 16385 T147 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[461 (1417794612583464960), 462 (1417794612586610688), 463 (1417794612594999296), 465 (1417794612596047873), 469 (1417794612601290753), 474 (1417794612604436482), 475 (1417794612609679362), 477 (1417794612614922240), 479 (1417794612629602304), 480 (1417794612639039489)]} 0 59
[junit4:junit4]   2> 16669 T81 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[535 (1417794612923203584), 536 (1417794612924252160), 537 (1417794612926349312), 540 (1417794612927397888), 544 (1417794612929495040), 548 (1417794612930543616), 549 (1417794612932640768), 550 (1417794612933689344), 552 (1417794612935786496), 553 (1417794612936835072)]} 0 15
[junit4:junit4]   2> 16720 T153 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[493 (1417794612607582208), 494 (1417794612613873665), 496 (1417794612614922240), 498 (1417794612622262272), 500 (1417794612624359424), 501 (1417794612626456576), 502 (1417794612628553728), 503 (1417794612629602304), 504 (1417794612640088065), 505 (1417794612642185217)]} 0 368
[junit4:junit4]   2> 16722 T139 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[511 (1417794612596047873), 512 (1417794612603387904), 513 (1417794612606533632), 515 (1417794612609679360), 517 (1417794612614922240), 519 (1417794612620165121), 520 (1417794612629602304), 521 (1417794612636942336), 522 (1417794612641136640), 523 (1417794612920057856)]} 0 380
[junit4:junit4]   2> 16728 T165 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[422 (1417794612566687744), 423 (1417794612585562112), 425 (1417794612600242178), 426 (1417794612608630784), 427 (1417794612612825088), 428 (1417794612619116547), 429 (1417794612623310848), 432 (1417794612628553729), 434 (1417794612639039488), 435 (1417794612642185216)]} 0 414
[junit4:junit4]   2> 16729 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[435 (1417794612567736320), 436 (1417794612569833473), 437 (1417794612572979200), 438 (1417794612578222081), 439 (1417794612593950723), 441 (1417794612598145024), 442 (1417794612602339328), 443 (1417794612609679361), 446 (1417794612640088067), 448 (1417794612990312449)]} 0 415
[junit4:junit4]   2> 16729 T136 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[384 (1417794612550959104), 388 (1417794612563542016), 392 (1417794612570882048), 394 (1417794612576124928), 397 (1417794612586610689), 398 (1417794612600242176), 399 (1417794612619116548), 401 (1417794612621213699), 402 (1417794612625408002), 403 (1417794612993458176)]} 0 431
[junit4:junit4]   2> 16726 T164 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[497 (1417794612589756416), 499 (1417794612590804992), 500 (1417794612599193600), 503 (1417794612601290752), 505 (1417794612605485056), 507 (1417794612608630785), 511 (1417794612611776513), 512 (1417794612621213697), 513 (1417794612625408000), 515 (1417794612640088064)]} 0 390
[junit4:junit4]   2> 16730 T154 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[466 (1417794612586610689), 470 (1417794612593950722), 471 (1417794612601290752), 475 (1417794612604436480), 476 (1417794612607582209), 482 (1417794612612825088), 484 (1417794612628553729), 488 (1417794612640088064), 489 (1417794612990312450), 490 (1417794612994506754)]} 0 400
[junit4:junit4]   2> 16734 T108 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[401 (1417794612565639168), 403 (1417794612571930625), 405 (1417794612575076352), 407 (1417794612579270656), 408 (1417794612592902146), 409 (1417794612598145025), 413 (1417794612606533633), 414 (1417794612609679360), 415 (1417794612637990912), 416 (1417794612997652480)]} 0 421
[junit4:junit4]   2> 16735 T106 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[419 (1417794612560396289), 420 (1417794612563542016), 424 (1417794612566687745), 428 (1417794612571930624), 429 (1417794612577173505), 430 (1417794612592902144), 431 (1417794612617019392), 432 (1417794612640088066), 433 (1417794612994506753), 434 (1417794613006041088)]} 0 427
[junit4:junit4]   2> 16735 T114 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[508 (1417794612623310848), 509 (1417794612625408000), 512 (1417794612627505152), 515 (1417794612632748032), 517 (1417794612635893760), 519 (1417794612637990913), 521 (1417794612642185216), 523 (1417794612994506752), 525 (1417794613002895361), 529 (1417794613006041090)]} 0 368
[junit4:junit4]   2> 16736 T120 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[449 (1417794612577173506), 452 (1417794612593950720), 453 (1417794612601290754), 454 (1417794612605485056), 457 (1417794612606533634), 458 (1417794612613873664), 462 (1417794612990312448), 463 (1417794612993458176), 464 (1417794613002895360), 465 (1417794613006041089)]} 0 412
[junit4:junit4]   2> 16742 T100 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[516 (1417794612619116546), 517 (1417794612622262272), 518 (1417794612624359425), 519 (1417794612627505152), 521 (1417794612633796608), 527 (1417794612636942336), 530 (1417794612995555328), 531 (1417794613001846784), 532 (1417794613006041088), 533 (1417794613008138240)]} 0 379
[junit4:junit4]   2> 16743 T52 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[509 (1417794612620165120), 511 (1417794612623310848), 512 (1417794612625408000), 513 (1417794612635893760), 517 (1417794612639039488), 518 (1417794612920057856), 519 (1417794612992409601), 520 (1417794613003943936), 521 (1417794613009186816), 524 (1417794613012332544)]} 0 378
[junit4:junit4]   2> 16739 T151 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[434 (1417794612589756416), 437 (1417794612604436480), 438 (1417794612608630784), 439 (1417794612611776512), 444 (1417794612615970817), 446 (1417794612619116544), 447 (1417794612640088064), 449 (1417794612991361024), 458 (1417794612998701056), 459 (1417794613002895361)]} 0 404
[junit4:junit4]   2> 16752 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[573 (1417794613007089665), 575 (1417794613009186816), 576 (1417794613011283969), 579 (1417794613012332545), 581 (1417794613014429696), 583 (1417794613016526848), 586 (1417794613017575424), 587 (1417794613018624001), 588 (1417794613020721152), 589 (1417794613021769728)]} 0 18
[junit4:junit4]   2> 16755 T103 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[526 (1417794612642185216), 528 (1417794612992409600), 529 (1417794612994506752), 531 (1417794613000798208), 532 (1417794613002895362), 533 (1417794613007089664), 535 (1417794613010235392), 536 (1417794613012332545), 539 (1417794613025964033), 544 (1417794613028061184)]} 0 370
[junit4:junit4]   2> 16758 T102 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[524 (1417794612922155008), 525 (1417794612996603904), 527 (1417794612999749632), 529 (1417794613001846784), 536 (1417794613007089664), 537 (1417794613009186816), 541 (1417794613012332544), 542 (1417794613022818304), 544 (1417794613027012608), 545 (1417794613029109762)]} 0 106
[junit4:junit4]   2> 16760 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[590 (1417794613008138241), 591 (1417794613011283968), 593 (1417794613012332544), 596 (1417794613013381121), 597 (1417794613022818304), 603 (1417794613023866880), 605 (1417794613024915456), 607 (1417794613028061184), 608 (1417794613030158336), 612 (1417794613032255488)]} 0 25
[junit4:junit4]   2> 16762 T78 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[463 (1417794612581367809), 466 (1417794612591853569), 472 (1417794612599193603), 473 (1417794612619116544), 474 (1417794612623310850), 475 (1417794612920057856), 476 (1417794613000798208), 477 (1417794613003943936), 478 (1417794613016526848), 481 (1417794613018624001)]} 0 434
[junit4:junit4]   2> 16808 T105 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[546 (1417794613001846785), 548 (1417794613006041088), 551 (1417794613008138240), 553 (1417794613011283968), 555 (1417794613021769728), 557 (1417794613024915456), 559 (1417794613029109760), 564 (1417794613031206914), 567 (1417794613035401216), 568 (1417794613082587136)]} 0 78
[junit4:junit4]   2> 16818 T66 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[610 (1417794613027012609), 611 (1417794613029109761), 612 (1417794613031206913), 613 (1417794613034352641), 617 (1417794613036449793), 623 (1417794613082587137), 629 (1417794613085732864), 632 (1417794613087830016), 634 (1417794613089927168), 636 (1417794613092024320)]} 0 65
[junit4:junit4]   2> 16820 T101 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[545 (1417794612999749632), 547 (1417794613002895360), 548 (1417794613011283968), 553 (1417794613014429696), 555 (1417794613017575424), 556 (1417794613018624000), 559 (1417794613020721152), 562 (1417794613025964032), 563 (1417794613029109760), 565 (1417794613095170048)]} 0 93
[junit4:junit4]   2> 16821 T129 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[569 (1417794613010235392), 572 (1417794613012332545), 574 (1417794613015478272), 575 (1417794613017575424), 577 (1417794613019672576), 580 (1417794613031206912), 581 (1417794613034352640), 586 (1417794613036449792), 589 (1417794613095170048), 591 (1417794613096218624)]} 0 84
[junit4:junit4]   2> 16822 T51 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[566 (1417794613027012608), 567 (1417794613029109761), 569 (1417794613032255488), 571 (1417794613081538560), 576 (1417794613083635712), 577 (1417794613084684289), 579 (1417794613086781440), 580 (1417794613087830017), 581 (1417794613089927168), 582 (1417794613092024320)]} 0 70
[junit4:junit4]   2> 16823 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[533 (1417794613001846784), 536 (1417794613006041091), 540 (1417794613008138240), 544 (1417794613010235392), 547 (1417794613016526849), 548 (1417794613018624000), 549 (1417794613021769729), 550 (1417794613023866881), 551 (1417794613030158337), 553 (1417794613034352641)]} 0 34
[junit4:junit4]   2> 16824 T122 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[601 (1417794613030158336), 603 (1417794613032255489), 605 (1417794613082587136), 608 (1417794613084684288), 609 (1417794613085732864), 611 (1417794613087830016), 613 (1417794613089927169), 616 (1417794613092024321), 617 (1417794613093072896), 619 (1417794613099364352)]} 0 68
[junit4:junit4]   2> 16899 T153 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[554 (1417794613007089664), 555 (1417794613009186817), 556 (1417794613011283970), 558 (1417794613013381120), 559 (1417794613022818305), 560 (1417794613029109760), 561 (1417794613032255489), 564 (1417794613035401216), 567 (1417794613101461504), 571 (1417794613102510081)]} 0 166
[junit4:junit4]   2> 16901 T100 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[575 (1417794613018624000), 576 (1417794613020721152), 577 (1417794613034352640), 578 (1417794613036449793), 579 (1417794613083635712), 583 (1417794613085732865), 584 (1417794613088878593), 585 (1417794613090975745), 588 (1417794613093072897), 589 (1417794613100412928)]} 0 157
[junit4:junit4]   2> 16906 T80 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[606 (1417794613081538560), 611 (1417794613084684288), 613 (1417794613086781440), 614 (1417794613088878592), 615 (1417794613090975744), 617 (1417794613093072896), 618 (1417794613099364352), 619 (1417794613101461504), 621 (1417794613102510082), 622 (1417794613180104704)]} 0 144
[junit4:junit4]   2> 16913 T145 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[583 (1417794613027012609), 586 (1417794613030158337), 587 (1417794613035401216), 589 (1417794613096218624), 591 (1417794613101461504), 595 (1417794613102510080), 596 (1417794613180104704), 597 (1417794613184299008), 599 (1417794613188493312), 600 (1417794613191639040)]} 0 160
[junit4:junit4]   2> 16917 T158 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[554 (1417794613013381120), 555 (1417794613024915456), 556 (1417794613029109760), 559 (1417794613031206912), 563 (1417794613036449792), 564 (1417794613102510080), 565 (1417794613187444736), 567 (1417794613190590464), 568 (1417794613192687616), 569 (1417794613193736192)]} 0 177
[junit4:junit4]   2> 16926 T150 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[592 (1417794613179056128), 593 (1417794613184299008), 594 (1417794613192687617), 595 (1417794613194784769), 598 (1417794613200027648), 600 (1417794613201076224), 604 (1417794613203173376), 606 (1417794613204221952), 608 (1417794613205270529), 609 (1417794613207367680)]} 0 101
[junit4:junit4]   2> 16932 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[697 (1417794613203173376), 698 (1417794613204221952), 700 (1417794613205270528), 703 (1417794613206319104), 704 (1417794613207367680), 705 (1417794613208416256), 706 (1417794613210513408), 707 (1417794613211561984), 710 (1417794613212610560), 711 (1417794613213659136)]} 0 12
[junit4:junit4]   2> 16933 T139 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[673 (1417794613200027649), 674 (1417794613201076225), 675 (1417794613203173377), 683 (1417794613205270528), 684 (1417794613206319104), 685 (1417794613208416256), 686 (1417794613209464832), 690 (1417794613211561984), 695 (1417794613212610560), 703 (1417794613214707712)]} 0 16
[junit4:junit4]   2> 16934 T147 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[620 (1417794613101461505), 622 (1417794613187444736), 624 (1417794613190590464), 625 (1417794613193736192), 626 (1417794613200027649), 630 (1417794613202124801), 634 (1417794613205270529), 637 (1417794613208416257), 638 (1417794613211561984), 641 (1417794613213659137)]} 0 172
[junit4:junit4]   2> 16935 T166 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[593 (1417794613027012608), 594 (1417794613035401216), 595 (1417794613082587136), 596 (1417794613085732864), 597 (1417794613087830016), 598 (1417794613089927168), 599 (1417794613092024320), 601 (1417794613099364353), 602 (1417794613102510081), 604 (1417794613186396160)]} 0 182
[junit4:junit4]   2> 16938 T128 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[757 (1417794613200027649), 758 (1417794613202124800), 759 (1417794613203173377), 764 (1417794613205270528), 766 (1417794613207367680), 768 (1417794613209464832), 772 (1417794613211561984), 773 (1417794613212610562), 775 (1417794613214707713), 776 (1417794613216804864)]} 0 20
[junit4:junit4]   2> 16939 T125 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[693 (1417794613203173376), 694 (1417794613204221953), 695 (1417794613206319105), 697 (1417794613208416257), 698 (1417794613210513409), 699 (1417794613212610560), 700 (1417794613214707712), 701 (1417794613215756289), 704 (1417794613217853441), 707 (1417794613219950592)]} 0 19
[junit4:junit4]   2> 16940 T161 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[665 (1417794613198979072), 667 (1417794613201076225), 671 (1417794613203173376), 675 (1417794613206319104), 678 (1417794613208416256), 680 (1417794613210513408), 682 (1417794613213659136), 684 (1417794613216804864), 686 (1417794613218902016), 688 (1417794613220999168)]} 0 25
[junit4:junit4]   2> 16940 T52 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[649 (1417794613200027648), 650 (1417794613202124800), 651 (1417794613205270528), 653 (1417794613207367680), 656 (1417794613209464832), 657 (1417794613212610560), 659 (1417794613214707712), 660 (1417794613217853441), 661 (1417794613219950592), 663 (1417794613220999169)]} 0 23
[junit4:junit4]   2> 16941 T112 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[778 (1417794613201076224), 779 (1417794613204221952), 780 (1417794613206319104), 781 (1417794613208416256), 782 (1417794613210513408), 784 (1417794613212610561), 785 (1417794613215756288), 787 (1417794613217853440), 788 (1417794613220999168), 789 (1417794613222047744)]} 0 22
[junit4:junit4]   2> 16952 T148 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[655 (1417794613186396160), 656 (1417794613187444736), 658 (1417794613189541888), 661 (1417794613190590464), 663 (1417794613192687616), 664 (1417794613194784768), 666 (1417794613227290624), 667 (1417794613230436352), 669 (1417794613232533504), 670 (1417794613233582082)]} 0 51
[junit4:junit4]   2> 16955 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[644 (1417794613180104704), 646 (1417794613184299008), 651 (1417794613185347585), 652 (1417794613192687617), 653 (1417794613194784768), 654 (1417794613227290625), 656 (1417794613232533504), 657 (1417794613234630656), 659 (1417794613235679235), 661 (1417794613237776384)]} 0 57
[junit4:junit4]   2> 16956 T117 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[727 (1417794613183250432), 729 (1417794613185347584), 732 (1417794613192687616), 737 (1417794613216804865), 739 (1417794613219950593), 740 (1417794613223096320), 741 (1417794613225193473), 742 (1417794613227290625), 747 (1417794613234630656), 750 (1417794613236727809)]} 0 54
[junit4:junit4]   2> 16958 T154 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[614 (1417794613034352640), 615 (1417794613100412928), 617 (1417794613187444736), 619 (1417794613189541888), 620 (1417794613191639041), 622 (1417794613219950593), 623 (1417794613222047745), 624 (1417794613225193472), 627 (1417794613227290624), 628 (1417794613235679232)]} 0 198
[junit4:junit4]   2> 16958 T160 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[753 (1417794613201076224), 757 (1417794613204221952), 759 (1417794613207367681), 761 (1417794613209464833), 763 (1417794613212610561), 764 (1417794613215756288), 765 (1417794613217853440), 766 (1417794613225193472), 767 (1417794613227290624), 768 (1417794613237776384)]} 0 40
[junit4:junit4]   2> 16962 T114 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[629 (1417794613191639040), 630 (1417794613194784769), 631 (1417794613228339201), 634 (1417794613235679233), 636 (1417794613236727808), 637 (1417794613237776386), 638 (1417794613239873536), 639 (1417794613240922113), 641 (1417794613244067840), 642 (1417794613245116416)]} 0 60
[junit4:junit4]   2> 16971 T133 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[647 (1417794613184299008), 648 (1417794613189541889), 651 (1417794613228339202), 652 (1417794613233582080), 653 (1417794613237776384), 657 (1417794613238824960), 658 (1417794613240922112), 661 (1417794613244067840), 662 (1417794613247213568), 664 (1417794613251407872)]} 0 69
[junit4:junit4]   2> 16971 T165 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[624 (1417794613095170048), 626 (1417794613097267200), 628 (1417794613178007552), 630 (1417794613184299009), 631 (1417794613195833344), 634 (1417794613218902016), 635 (1417794613222047745), 636 (1417794613224144896), 639 (1417794613226242048), 641 (1417794613228339201)]} 0 153
[junit4:junit4]   2> 16979 T108 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[663 (1417794613082587136), 664 (1417794613102510080), 666 (1417794613179056128), 667 (1417794613183250433), 668 (1417794613185347584), 669 (1417794613192687616), 671 (1417794613193736192), 672 (1417794613195833344), 673 (1417794613255602176), 674 (1417794613256650752)]} 0 174
[junit4:junit4]   2> 16980 T146 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[637 (1417794613230436353), 638 (1417794613241970690), 639 (1417794613245116417), 640 (1417794613246164993), 645 (1417794613253505024), 646 (1417794613255602176), 647 (1417794613257699328), 648 (1417794613258747904), 649 (1417794613262942208), 651 (1417794613263990785)]} 0 34
[junit4:junit4]   2> 16981 T67 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[726 (1417794613233582081), 729 (1417794613236727809), 730 (1417794613237776384), 733 (1417794613239873536), 735 (1417794613241970689), 738 (1417794613245116416), 739 (1417794613247213568), 740 (1417794613255602177), 741 (1417794613259796481), 742 (1417794613263990784)]} 0 31
[junit4:junit4]   2> 16979 T120 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[714 (1417794613194784770), 716 (1417794613217853441), 717 (1417794613219950592), 718 (1417794613222047744), 719 (1417794613223096320), 720 (1417794613225193473), 722 (1417794613226242049), 725 (1417794613233582080), 727 (1417794613240922114), 728 (1417794613258747905)]} 0 66
[junit4:junit4]   2> 16983 T132 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[665 (1417794613200027648), 670 (1417794613218902017), 673 (1417794613220999169), 674 (1417794613222047746), 678 (1417794613225193472), 679 (1417794613227290624), 682 (1417794613228339200), 685 (1417794613255602176), 686 (1417794613258747904), 690 (1417794613265039360)]} 0 66
[junit4:junit4]   2> 16983 T151 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[769 (1417794613233582080), 770 (1417794613235679232), 771 (1417794613236727808), 774 (1417794613237776385), 776 (1417794613240922113), 777 (1417794613241970688), 780 (1417794613245116417), 781 (1417794613247213568), 782 (1417794613256650752), 784 (1417794613265039361)]} 0 33
[junit4:junit4]   2> 16984 T65 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[705 (1417794613232533505), 707 (1417794613236727808), 708 (1417794613238824960), 710 (1417794613240922112), 713 (1417794613249310720), 717 (1417794613251407872), 719 (1417794613252456448), 720 (1417794613254553600), 721 (1417794613258747905), 723 (1417794613265039360)]} 0 36
[junit4:junit4]   2> 16984 T164 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[724 (1417794613189541888), 725 (1417794613190590465), 726 (1417794613234630656), 727 (1417794613236727808), 728 (1417794613237776385), 730 (1417794613247213569), 732 (1417794613250359297), 733 (1417794613252456448), 737 (1417794613256650752), 738 (1417794613262942209)]} 0 76
[junit4:junit4]   2> 16987 T136 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[739 (1417794613235679232), 740 (1417794613239873536), 741 (1417794613241970688), 744 (1417794613246164992), 745 (1417794613250359296), 746 (1417794613252456449), 749 (1417794613254553600), 752 (1417794613257699328), 754 (1417794613265039362), 756 (1417794613270282240)]} 0 36
[junit4:junit4]   2> 16999 T149 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[746 (1417794613233582080), 748 (1417794613236727810), 749 (1417794613239873537), 752 (1417794613241970688), 754 (1417794613244067840), 756 (1417794613246164992), 760 (1417794613254553601), 763 (1417794613259796480), 764 (1417794613263990786), 767 (1417794613266087937)]} 0 49
[junit4:junit4]   2> 17001 T106 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[677 (1417794613183250432), 680 (1417794613188493312), 683 (1417794613190590464), 687 (1417794613217853440), 688 (1417794613220999168), 689 (1417794613224144896), 691 (1417794613226242048), 693 (1417794613228339200), 694 (1417794613267136514), 696 (1417794613271330818)]} 0 102
[junit4:junit4]   2> 17002 T126 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[754 (1417794613235679234), 756 (1417794613246164992), 759 (1417794613247213568), 762 (1417794613250359297), 769 (1417794613252456448), 770 (1417794613255602177), 773 (1417794613258747904), 775 (1417794613262942208), 776 (1417794613265039360), 781 (1417794613271330816)]} 0 50
[junit4:junit4]   2> 17003 T103 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[785 (1417794613252456448), 788 (1417794613254553600), 790 (1417794613262942208), 792 (1417794613263990784), 793 (1417794613266087936), 796 (1417794613267136514), 798 (1417794613270282240), 800 (1417794613272379392), 801 (1417794613284962304), 803 (1417794613287059456)]} 0 35
[junit4:junit4]   2> 17002 T153 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[734 (1417794613230436352), 739 (1417794613237776385), 740 (1417794613240922112), 741 (1417794613250359296), 743 (1417794613257699328), 744 (1417794613259796480), 747 (1417794613263990785), 749 (1417794613267136512), 750 (1417794613270282241), 752 (1417794613272379393)]} 0 55
[junit4:junit4]   2> 17005 T81 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[708 (1417794613256650753), 709 (1417794613262942208), 710 (1417794613263990784), 712 (1417794613265039361), 714 (1417794613267136512), 715 (1417794613269233664), 716 (1417794613271330817), 718 (1417794613272379392), 720 (1417794613284962304), 722 (1417794613286010880)]} 0 45
[junit4:junit4]   2> 17006 T53 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[708 (1417794613240922112), 710 (1417794613245116416), 714 (1417794613246164992), 716 (1417794613253505024), 719 (1417794613260845056), 720 (1417794613265039360), 722 (1417794613267136513), 723 (1417794613271330816), 724 (1417794613286010880), 725 (1417794613289156608)]} 0 50
[junit4:junit4]   2> 17005 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[782 (1417794613251407872), 785 (1417794613253505024), 786 (1417794613258747906), 787 (1417794613266087936), 789 (1417794613268185088), 791 (1417794613270282240), 792 (1417794613283913728), 793 (1417794613287059456), 794 (1417794613289156609), 795 (1417794613290205184)]} 0 38
[junit4:junit4]   2> 17007 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[796 (1417794613263990784), 797 (1417794613267136513), 798 (1417794613269233664), 799 (1417794613272379392), 801 (1417794613284962304), 804 (1417794613286010881), 806 (1417794613287059458), 809 (1417794613289156610), 813 (1417794613290205186), 816 (1417794613292302337)]} 0 29
[junit4:junit4]   2> 17010 T37 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[817 (1417794613267136515), 821 (1417794613271330817), 822 (1417794613283913729), 824 (1417794613286010880), 826 (1417794613287059457), 828 (1417794613289156608), 829 (1417794613290205185), 830 (1417794613292302336), 831 (1417794613293350912), 832 (1417794613294399488)]} 0 29
[junit4:junit4]   2> 17011 T129 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[799 (1417794613265039361), 800 (1417794613269233664), 801 (1417794613271330816), 804 (1417794613284962304), 808 (1417794613287059456), 809 (1417794613289156608), 811 (1417794613291253761), 812 (1417794613292302337), 814 (1417794613294399488), 823 (1417794613295448065)]} 0 30
[junit4:junit4]   2> 17011 T127 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[689 (1417794613231484928), 690 (1417794613232533504), 695 (1417794613244067840), 696 (1417794613257699328), 697 (1417794613267136512), 698 (1417794613268185088), 702 (1417794613272379393), 705 (1417794613286010881), 706 (1417794613293350912), 707 (1417794613295448065)]} 0 64
[junit4:junit4]   2> 17012 T102 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[768 (1417794613262942209), 777 (1417794613266087936), 779 (1417794613268185088), 784 (1417794613288108032), 786 (1417794613289156609), 788 (1417794613291253760), 789 (1417794613292302336), 790 (1417794613293350912), 792 (1417794613295448064), 795 (1417794613296496640)]} 0 34
[junit4:junit4]   2> 17018 T78 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[790 (1417794613262942210), 791 (1417794613271330816), 796 (1417794613284962305), 797 (1417794613287059456), 798 (1417794613289156608), 800 (1417794613291253760), 806 (1417794613294399488), 811 (1417794613296496640), 812 (1417794613299642368), 817 (1417794613301739520)]} 0 40
[junit4:junit4]   2> 17030 T39 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[833 (1417794613295448064), 837 (1417794613297545216), 843 (1417794613298593792), 844 (1417794613304885249), 845 (1417794613306982401), 848 (1417794613309079552), 852 (1417794613310128129), 855 (1417794613312225280), 857 (1417794613313273856), 858 (1417794613315371008)]} 0 21
[junit4:junit4]   2> 17032 T105 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[849 (1417794613297545216), 850 (1417794613299642368), 855 (1417794613300690944), 857 (1417794613302788096), 859 (1417794613303836672), 860 (1417794613305933824), 863 (1417794613309079554), 864 (1417794613311176704), 867 (1417794613313273856), 871 (1417794613315371008)]} 0 22
[junit4:junit4]   2> 17042 T158 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[836 (1417794613300690944), 839 (1417794613303836672), 840 (1417794613305933825), 843 (1417794613314322432), 844 (1417794613317468160), 845 (1417794613319565312), 849 (1417794613321662466), 850 (1417794613323759617), 851 (1417794613325856769), 852 (1417794613327953921)]} 0 29
[junit4:junit4]   2> 17043 T114 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[859 (1417794613299642368), 863 (1417794613300690944), 867 (1417794613302788096), 870 (1417794613303836672), 871 (1417794613305933824), 873 (1417794613309079553), 874 (1417794613311176704), 875 (1417794613312225281), 876 (1417794613325856768), 878 (1417794613327953920)]} 0 31
[junit4:junit4]   2> 17046 T101 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[804 (1417794613300690945), 805 (1417794613305933824), 806 (1417794613308030976), 809 (1417794613314322433), 812 (1417794613318516736), 814 (1417794613321662464), 815 (1417794613324808192), 816 (1417794613326905344), 817 (1417794613329002496), 818 (1417794613332148224)]} 0 33
[junit4:junit4]   2> 17047 T127 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[819 (1417794613300690944), 820 (1417794613302788096), 823 (1417794613305933825), 827 (1417794613309079552), 828 (1417794613317468162), 829 (1417794613319565312), 830 (1417794613322711041), 832 (1417794613324808194), 833 (1417794613327953921), 837 (1417794613332148225)]} 0 34
[junit4:junit4]   2> 17057 T66 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[825 (1417794613295448066), 827 (1417794613297545217), 831 (1417794613299642370), 835 (1417794613301739520), 838 (1417794613304885248), 839 (1417794613306982400), 841 (1417794613309079553), 842 (1417794613334245376), 846 (1417794613340536833), 847 (1417794613342633985)]} 0 48
[junit4:junit4]   2> 17059 T126 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[930 (1417794613317468160), 931 (1417794613319565312), 935 (1417794613320613888), 936 (1417794613322711040), 938 (1417794613332148225), 939 (1417794613334245377), 940 (1417794613341585409), 942 (1417794613343682560), 943 (1417794613344731137), 945 (1417794613345779713)]} 0 29
[junit4:junit4]   2> 17060 T161 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[904 (1417794613317468160), 906 (1417794613320613888), 908 (1417794613322711040), 909 (1417794613326905345), 912 (1417794613330051072), 922 (1417794613335293952), 923 (1417794613337391104), 925 (1417794613339488256), 926 (1417794613341585408), 928 (1417794613343682561)]} 0 30
[junit4:junit4]   2> 17061 T146 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[920 (1417794613320613888), 921 (1417794613323759616), 922 (1417794613324808192), 923 (1417794613326905344), 924 (1417794613327953920), 925 (1417794613330051072), 927 (1417794613335293952), 928 (1417794613336342528), 931 (1417794613337391104), 932 (1417794613339488257)]} 0 28
[junit4:junit4]   2> 17062 T112 C4 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[916 (1417794613318516736), 917 (1417794613321662465), 920 (1417794613323759616), 922 (1417794613325856768), 923 (1417794613327953920), 929 (1417794613330051072), 930 (1417794613332148224), 932 (1417794613334245376), 933 (1417794613342633984), 938 (1417794613344731136)]} 0 32
[junit4:junit4]   2> 17062 T51 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[843 (1417794613295448064), 844 (1417794613297545216), 845 (1417794613299642368), 850 (1417794613301739520), 851 (1417794613306982400), 853 (1417794613311176704), 854 (1417794613314322432), 855 (1417794613316419584), 862 (1417794613340536832), 863 (1417794613347876865)]} 0 54
[junit4:junit4]   2> 17064 T153 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[946 (1417794613323759616), 947 (1417794613324808192), 948 (1417794613330051072), 950 (1417794613332148224), 951 (1417794613339488256), 952 (1417794613340536832), 953 (1417794613342633984), 954 (1417794613347876864), 955 (1417794613349974016), 956 (1417794613351022593)]} 0 29
[junit4:junit4]   2> 17066 T150 C3 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[906 (1417794613323759617), 907 (1417794613325856768), 908 (1417794613329002496), 909 (1417794613331099648), 910 (1417794613333196800), 911 (1417794613341585409), 912 (1417794613344731137), 913 (1417794613346828290), 914 (1417794613348925440), 918 (1417794613351022593)]} 0 39
[junit4:junit4]   2> 17067 T38 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[957 (1417794613334245376), 958 (1417794613341585408), 959 (1417794613344731136), 960 (1417794613345779712), 962 (1417794613346828291), 963 (1417794613348925440), 965 (1417794613349974017), 966 (1417794613352071168), 967 (1417794613353119746), 968 (1417794613355216897)]} 0 22
[junit4:junit4]   2> 17069 T108 C1 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[879 (1417794613301739520), 880 (1417794613304885248), 881 (1417794613306982400), 883 (1417794613310128128), 884 (1417794613316419584), 885 (1417794613343682561), 888 (1417794613346828288), 892 (1417794613351022592), 893 (1417794613355216896), 896 (1417794613356265473)]} 0 54
[junit4:junit4]   2> 17070 T117 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[955 (1417794613335293953), 957 (1417794613338439680), 959 (1417794613340536833), 961 (1417794613342633985), 962 (1417794613344731136), 964 (1417794613346828288), 966 (1417794613348925441), 972 (1417794613354168320), 973 (1417794613356265473), 975 (1417794613357314050)]} 0 23
[junit4:junit4]   2> 17070 T122 C2 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[866 (1417794613303836672), 870 (1417794613306982401), 872 (1417794613310128128), 873 (1417794613339488258), 874 (1417794613343682560), 876 (1417794613345779712), 877 (1417794613348925440), 878 (1417794613352071169), 881 (1417794613354168321), 882 (1417794613357314048)]} 0 53
[junit4:junit4]   2> 17074 T148 C3 UPDATE [collection1] webapp=/solr path

[...truncated too long message...]

 at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  24) Thread[id=54, name=qtp1989367070-54 Acceptor3 SelectChannelConnector@0.0.0.0:16252, state=BLOCKED, group=TGRP-TestDistributedSearch]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  25) Thread[id=36, name=qtp1286237868-36 Acceptor2 SelectChannelConnector@0.0.0.0:60522, state=RUNNABLE, group=TGRP-TestDistributedSearch]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  26) Thread[id=141, name=qtp536825004-141 Acceptor1 SelectChannelConnector@0.0.0.0:16272, state=BLOCKED, group=TGRP-TestDistributedSearch]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  27) Thread[id=68, name=qtp629076881-68 Acceptor3 SelectChannelConnector@0.0.0.0:16255, state=BLOCKED, group=TGRP-TestDistributedSearch]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  28) Thread[id=78, name=qtp736059716-78 Acceptor0 SelectChannelConnector@0.0.0.0:16256, state=RUNNABLE, group=TGRP-TestDistributedSearch]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  29) Thread[id=102, name=qtp779071350-102 Acceptor2 SelectChannelConnector@0.0.0.0:16267, state=RUNNABLE, group=TGRP-TestDistributedSearch]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  30) Thread[id=53, name=qtp1989367070-53 Acceptor2 SelectChannelConnector@0.0.0.0:16252, state=RUNNABLE, group=TGRP-TestDistributedSearch]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  31) Thread[id=51, name=qtp1989367070-51 Acceptor0 SelectChannelConnector@0.0.0.0:16252, state=BLOCKED, group=TGRP-TestDistributedSearch]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  32) Thread[id=23, name=qtp1412500084-23 Acceptor0 SelectChannelConnector@0.0.0.0:28646, state=BLOCKED, group=TGRP-TestDistributedSearch]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Compressing(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=408)), sim=DefaultSimilarity, locale=ar_YE, timezone=Europe/Budapest
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=33,free=100213968,total=125829120
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestSolrXMLSerializer, TestDistributedSearch]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDistributedSearch -Dtests.seed=43BDC8EDB8C9CCE8 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar_YE -Dtests.timezone=Europe/Budapest -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] ERROR   0.00s J1 | TestDistributedSearch (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([43BDC8EDB8C9CCE8]:0)
[junit4:junit4] Completed on J1 in 7224.04s, 1 test, 2 errors <<< FAILURES!

[...truncated 183 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:335: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/module-build.xml:61: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:1163: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:827: There were test failures: 233 suites, 965 tests, 3 suite-level errors, 1 error, 879 ignored

Total time: 138 minutes 25 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 610 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/610/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.RecoveryZkTest

Error Message:
8 threads leaked from SUITE scope at org.apache.solr.cloud.RecoveryZkTest:     1) Thread[id=27, name=qtp806943770-27 Acceptor0 SelectChannelConnector@0.0.0.0:19540, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=30, name=qtp806943770-30 Acceptor3 SelectChannelConnector@0.0.0.0:19540, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=104, name=qtp1442633686-104 Acceptor1 SelectChannelConnector@0.0.0.0:17211, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=105, name=qtp1442633686-105 Acceptor2 SelectChannelConnector@0.0.0.0:17211, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=106, name=qtp1442633686-106 Acceptor3 SelectChannelConnector@0.0.0.0:17211, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=103, name=qtp1442633686-103 Acceptor0 SelectChannelConnector@0.0.0.0:17211, state=RUNNABLE, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=28, name=qtp806943770-28 Acceptor1 SelectChannelConnector@0.0.0.0:19540, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=29, name=qtp806943770-29 Acceptor2 SelectChannelConnector@0.0.0.0:19540, state=RUNNABLE, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 8 threads leaked from SUITE scope at org.apache.solr.cloud.RecoveryZkTest: 
   1) Thread[id=27, name=qtp806943770-27 Acceptor0 SelectChannelConnector@0.0.0.0:19540, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=30, name=qtp806943770-30 Acceptor3 SelectChannelConnector@0.0.0.0:19540, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=104, name=qtp1442633686-104 Acceptor1 SelectChannelConnector@0.0.0.0:17211, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=105, name=qtp1442633686-105 Acceptor2 SelectChannelConnector@0.0.0.0:17211, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=106, name=qtp1442633686-106 Acceptor3 SelectChannelConnector@0.0.0.0:17211, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=103, name=qtp1442633686-103 Acceptor0 SelectChannelConnector@0.0.0.0:17211, state=RUNNABLE, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=28, name=qtp806943770-28 Acceptor1 SelectChannelConnector@0.0.0.0:19540, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=29, name=qtp806943770-29 Acceptor2 SelectChannelConnector@0.0.0.0:19540, state=RUNNABLE, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([8573C2ED46DC4A5E]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.RecoveryZkTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=27, name=qtp806943770-27 Acceptor0 SelectChannelConnector@0.0.0.0:19540, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=30, name=qtp806943770-30 Acceptor3 SelectChannelConnector@0.0.0.0:19540, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=104, name=qtp1442633686-104 Acceptor1 SelectChannelConnector@0.0.0.0:17211, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=105, name=qtp1442633686-105 Acceptor2 SelectChannelConnector@0.0.0.0:17211, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=106, name=qtp1442633686-106 Acceptor3 SelectChannelConnector@0.0.0.0:17211, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=103, name=qtp1442633686-103 Acceptor0 SelectChannelConnector@0.0.0.0:17211, state=RUNNABLE, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=28, name=qtp806943770-28 Acceptor1 SelectChannelConnector@0.0.0.0:19540, state=BLOCKED, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=29, name=qtp806943770-29 Acceptor2 SelectChannelConnector@0.0.0.0:19540, state=RUNNABLE, group=TGRP-RecoveryZkTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=27, name=qtp806943770-27 Acceptor0 SelectChannelConnector@0.0.0.0:19540, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=30, name=qtp806943770-30 Acceptor3 SelectChannelConnector@0.0.0.0:19540, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=104, name=qtp1442633686-104 Acceptor1 SelectChannelConnector@0.0.0.0:17211, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=105, name=qtp1442633686-105 Acceptor2 SelectChannelConnector@0.0.0.0:17211, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=106, name=qtp1442633686-106 Acceptor3 SelectChannelConnector@0.0.0.0:17211, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=103, name=qtp1442633686-103 Acceptor0 SelectChannelConnector@0.0.0.0:17211, state=RUNNABLE, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=28, name=qtp806943770-28 Acceptor1 SelectChannelConnector@0.0.0.0:19540, state=BLOCKED, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=29, name=qtp806943770-29 Acceptor2 SelectChannelConnector@0.0.0.0:19540, state=RUNNABLE, group=TGRP-RecoveryZkTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([8573C2ED46DC4A5E]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.component.DistributedSpellCheckComponentTest

Error Message:
36 threads leaked from SUITE scope at org.apache.solr.handler.component.DistributedSpellCheckComponentTest:     1) Thread[id=188, name=qtp109855678-188 Acceptor1 SelectChannelConnector@0.0.0.0:19714, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=153, name=qtp1942734199-153 Acceptor1 SelectChannelConnector@0.0.0.0:10507, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=142, name=qtp1432551238-142 Acceptor3 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=174, name=qtp1979690111-174 Acceptor1 SelectChannelConnector@0.0.0.0:15460, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=202, name=qtp406322006-202 Acceptor1 SelectChannelConnector@0.0.0.0:30562, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=155, name=qtp1942734199-155 Acceptor3 SelectChannelConnector@0.0.0.0:10507, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=204, name=qtp406322006-204 Acceptor3 SelectChannelConnector@0.0.0.0:30562, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=215, name=qtp453856425-215 Acceptor0 SelectChannelConnector@0.0.0.0:25011, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=97, name=qtp1431523121-97 Acceptor1 SelectChannelConnector@0.0.0.0:19597, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=99, name=qtp1431523121-99 Acceptor3 SelectChannelConnector@0.0.0.0:19597, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=187, name=qtp109855678-187 Acceptor0 SelectChannelConnector@0.0.0.0:19714, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=216, name=qtp453856425-216 Acceptor1 SelectChannelConnector@0.0.0.0:25011, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=129, name=qtp390636756-129 Acceptor3 SelectChannelConnector@0.0.0.0:51378, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=112, name=qtp1507644908-112 Acceptor2 SelectChannelConnector@0.0.0.0:28779, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=201, name=qtp406322006-201 Acceptor0 SelectChannelConnector@0.0.0.0:30562, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=176, name=qtp1979690111-176 Acceptor3 SelectChannelConnector@0.0.0.0:15460, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=98, name=qtp1431523121-98 Acceptor2 SelectChannelConnector@0.0.0.0:19597, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=175, name=qtp1979690111-175 Acceptor2 SelectChannelConnector@0.0.0.0:15460, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=126, name=qtp390636756-126 Acceptor0 SelectChannelConnector@0.0.0.0:51378, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=113, name=qtp1507644908-113 Acceptor3 SelectChannelConnector@0.0.0.0:28779, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   21) Thread[id=190, name=qtp109855678-190 Acceptor3 SelectChannelConnector@0.0.0.0:19714, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   22) Thread[id=111, name=qtp1507644908-111 Acceptor1 SelectChannelConnector@0.0.0.0:28779, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   23) Thread[id=189, name=qtp109855678-189 Acceptor2 SelectChannelConnector@0.0.0.0:19714, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   24) Thread[id=217, name=qtp453856425-217 Acceptor2 SelectChannelConnector@0.0.0.0:25011, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   25) Thread[id=173, name=qtp1979690111-173 Acceptor0 SelectChannelConnector@0.0.0.0:15460, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   26) Thread[id=152, name=qtp1942734199-152 Acceptor0 SelectChannelConnector@0.0.0.0:10507, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   27) Thread[id=203, name=qtp406322006-203 Acceptor2 SelectChannelConnector@0.0.0.0:30562, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   28) Thread[id=127, name=qtp390636756-127 Acceptor1 SelectChannelConnector@0.0.0.0:51378, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   29) Thread[id=110, name=qtp1507644908-110 Acceptor0 SelectChannelConnector@0.0.0.0:28779, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   30) Thread[id=154, name=qtp1942734199-154 Acceptor2 SelectChannelConnector@0.0.0.0:10507, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   31) Thread[id=96, name=qtp1431523121-96 Acceptor0 SelectChannelConnector@0.0.0.0:19597, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   32) Thread[id=140, name=qtp1432551238-140 Acceptor1 SelectChannelConnector@0.0.0.0:49279, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   33) Thread[id=218, name=qtp453856425-218 Acceptor3 SelectChannelConnector@0.0.0.0:25011, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   34) Thread[id=128, name=qtp390636756-128 Acceptor2 SelectChannelConnector@0.0.0.0:51378, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   35) Thread[id=141, name=qtp1432551238-141 Acceptor2 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   36) Thread[id=139, name=qtp1432551238-139 Acceptor0 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 36 threads leaked from SUITE scope at org.apache.solr.handler.component.DistributedSpellCheckComponentTest: 
   1) Thread[id=188, name=qtp109855678-188 Acceptor1 SelectChannelConnector@0.0.0.0:19714, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=153, name=qtp1942734199-153 Acceptor1 SelectChannelConnector@0.0.0.0:10507, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=142, name=qtp1432551238-142 Acceptor3 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=174, name=qtp1979690111-174 Acceptor1 SelectChannelConnector@0.0.0.0:15460, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=202, name=qtp406322006-202 Acceptor1 SelectChannelConnector@0.0.0.0:30562, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=155, name=qtp1942734199-155 Acceptor3 SelectChannelConnector@0.0.0.0:10507, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=204, name=qtp406322006-204 Acceptor3 SelectChannelConnector@0.0.0.0:30562, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=215, name=qtp453856425-215 Acceptor0 SelectChannelConnector@0.0.0.0:25011, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=97, name=qtp1431523121-97 Acceptor1 SelectChannelConnector@0.0.0.0:19597, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=99, name=qtp1431523121-99 Acceptor3 SelectChannelConnector@0.0.0.0:19597, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=187, name=qtp109855678-187 Acceptor0 SelectChannelConnector@0.0.0.0:19714, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=216, name=qtp453856425-216 Acceptor1 SelectChannelConnector@0.0.0.0:25011, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=129, name=qtp390636756-129 Acceptor3 SelectChannelConnector@0.0.0.0:51378, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=112, name=qtp1507644908-112 Acceptor2 SelectChannelConnector@0.0.0.0:28779, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=201, name=qtp406322006-201 Acceptor0 SelectChannelConnector@0.0.0.0:30562, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=176, name=qtp1979690111-176 Acceptor3 SelectChannelConnector@0.0.0.0:15460, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=98, name=qtp1431523121-98 Acceptor2 SelectChannelConnector@0.0.0.0:19597, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=175, name=qtp1979690111-175 Acceptor2 SelectChannelConnector@0.0.0.0:15460, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=126, name=qtp390636756-126 Acceptor0 SelectChannelConnector@0.0.0.0:51378, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=113, name=qtp1507644908-113 Acceptor3 SelectChannelConnector@0.0.0.0:28779, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  21) Thread[id=190, name=qtp109855678-190 Acceptor3 SelectChannelConnector@0.0.0.0:19714, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  22) Thread[id=111, name=qtp1507644908-111 Acceptor1 SelectChannelConnector@0.0.0.0:28779, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  23) Thread[id=189, name=qtp109855678-189 Acceptor2 SelectChannelConnector@0.0.0.0:19714, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  24) Thread[id=217, name=qtp453856425-217 Acceptor2 SelectChannelConnector@0.0.0.0:25011, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  25) Thread[id=173, name=qtp1979690111-173 Acceptor0 SelectChannelConnector@0.0.0.0:15460, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  26) Thread[id=152, name=qtp1942734199-152 Acceptor0 SelectChannelConnector@0.0.0.0:10507, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  27) Thread[id=203, name=qtp406322006-203 Acceptor2 SelectChannelConnector@0.0.0.0:30562, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  28) Thread[id=127, name=qtp390636756-127 Acceptor1 SelectChannelConnector@0.0.0.0:51378, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  29) Thread[id=110, name=qtp1507644908-110 Acceptor0 SelectChannelConnector@0.0.0.0:28779, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  30) Thread[id=154, name=qtp1942734199-154 Acceptor2 SelectChannelConnector@0.0.0.0:10507, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  31) Thread[id=96, name=qtp1431523121-96 Acceptor0 SelectChannelConnector@0.0.0.0:19597, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  32) Thread[id=140, name=qtp1432551238-140 Acceptor1 SelectChannelConnector@0.0.0.0:49279, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  33) Thread[id=218, name=qtp453856425-218 Acceptor3 SelectChannelConnector@0.0.0.0:25011, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  34) Thread[id=128, name=qtp390636756-128 Acceptor2 SelectChannelConnector@0.0.0.0:51378, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  35) Thread[id=141, name=qtp1432551238-141 Acceptor2 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  36) Thread[id=139, name=qtp1432551238-139 Acceptor0 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([8573C2ED46DC4A5E]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.component.DistributedSpellCheckComponentTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=188, name=qtp109855678-188 Acceptor1 SelectChannelConnector@0.0.0.0:19714, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=153, name=qtp1942734199-153 Acceptor1 SelectChannelConnector@0.0.0.0:10507, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=142, name=qtp1432551238-142 Acceptor3 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=174, name=qtp1979690111-174 Acceptor1 SelectChannelConnector@0.0.0.0:15460, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    5) Thread[id=202, name=qtp406322006-202 Acceptor1 SelectChannelConnector@0.0.0.0:30562, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    6) Thread[id=155, name=qtp1942734199-155 Acceptor3 SelectChannelConnector@0.0.0.0:10507, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    7) Thread[id=204, name=qtp406322006-204 Acceptor3 SelectChannelConnector@0.0.0.0:30562, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    8) Thread[id=215, name=qtp453856425-215 Acceptor0 SelectChannelConnector@0.0.0.0:25011, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    9) Thread[id=97, name=qtp1431523121-97 Acceptor1 SelectChannelConnector@0.0.0.0:19597, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   10) Thread[id=99, name=qtp1431523121-99 Acceptor3 SelectChannelConnector@0.0.0.0:19597, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   11) Thread[id=187, name=qtp109855678-187 Acceptor0 SelectChannelConnector@0.0.0.0:19714, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   12) Thread[id=216, name=qtp453856425-216 Acceptor1 SelectChannelConnector@0.0.0.0:25011, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   13) Thread[id=129, name=qtp390636756-129 Acceptor3 SelectChannelConnector@0.0.0.0:51378, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   14) Thread[id=112, name=qtp1507644908-112 Acceptor2 SelectChannelConnector@0.0.0.0:28779, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   15) Thread[id=176, name=qtp1979690111-176 Acceptor3 SelectChannelConnector@0.0.0.0:15460, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   16) Thread[id=201, name=qtp406322006-201 Acceptor0 SelectChannelConnector@0.0.0.0:30562, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   17) Thread[id=98, name=qtp1431523121-98 Acceptor2 SelectChannelConnector@0.0.0.0:19597, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   18) Thread[id=175, name=qtp1979690111-175 Acceptor2 SelectChannelConnector@0.0.0.0:15460, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   19) Thread[id=113, name=qtp1507644908-113 Acceptor3 SelectChannelConnector@0.0.0.0:28779, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   20) Thread[id=126, name=qtp390636756-126 Acceptor0 SelectChannelConnector@0.0.0.0:51378, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   21) Thread[id=111, name=qtp1507644908-111 Acceptor1 SelectChannelConnector@0.0.0.0:28779, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   22) Thread[id=190, name=qtp109855678-190 Acceptor3 SelectChannelConnector@0.0.0.0:19714, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   23) Thread[id=189, name=qtp109855678-189 Acceptor2 SelectChannelConnector@0.0.0.0:19714, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   24) Thread[id=217, name=qtp453856425-217 Acceptor2 SelectChannelConnector@0.0.0.0:25011, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   25) Thread[id=173, name=qtp1979690111-173 Acceptor0 SelectChannelConnector@0.0.0.0:15460, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   26) Thread[id=152, name=qtp1942734199-152 Acceptor0 SelectChannelConnector@0.0.0.0:10507, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   27) Thread[id=203, name=qtp406322006-203 Acceptor2 SelectChannelConnector@0.0.0.0:30562, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   28) Thread[id=127, name=qtp390636756-127 Acceptor1 SelectChannelConnector@0.0.0.0:51378, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   29) Thread[id=110, name=qtp1507644908-110 Acceptor0 SelectChannelConnector@0.0.0.0:28779, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   30) Thread[id=154, name=qtp1942734199-154 Acceptor2 SelectChannelConnector@0.0.0.0:10507, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   31) Thread[id=96, name=qtp1431523121-96 Acceptor0 SelectChannelConnector@0.0.0.0:19597, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   32) Thread[id=140, name=qtp1432551238-140 Acceptor1 SelectChannelConnector@0.0.0.0:49279, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   33) Thread[id=218, name=qtp453856425-218 Acceptor3 SelectChannelConnector@0.0.0.0:25011, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   34) Thread[id=128, name=qtp390636756-128 Acceptor2 SelectChannelConnector@0.0.0.0:51378, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   35) Thread[id=141, name=qtp1432551238-141 Acceptor2 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)   36) Thread[id=139, name=qtp1432551238-139 Acceptor0 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=188, name=qtp109855678-188 Acceptor1 SelectChannelConnector@0.0.0.0:19714, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=153, name=qtp1942734199-153 Acceptor1 SelectChannelConnector@0.0.0.0:10507, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=142, name=qtp1432551238-142 Acceptor3 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=174, name=qtp1979690111-174 Acceptor1 SelectChannelConnector@0.0.0.0:15460, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   5) Thread[id=202, name=qtp406322006-202 Acceptor1 SelectChannelConnector@0.0.0.0:30562, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   6) Thread[id=155, name=qtp1942734199-155 Acceptor3 SelectChannelConnector@0.0.0.0:10507, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   7) Thread[id=204, name=qtp406322006-204 Acceptor3 SelectChannelConnector@0.0.0.0:30562, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   8) Thread[id=215, name=qtp453856425-215 Acceptor0 SelectChannelConnector@0.0.0.0:25011, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   9) Thread[id=97, name=qtp1431523121-97 Acceptor1 SelectChannelConnector@0.0.0.0:19597, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  10) Thread[id=99, name=qtp1431523121-99 Acceptor3 SelectChannelConnector@0.0.0.0:19597, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  11) Thread[id=187, name=qtp109855678-187 Acceptor0 SelectChannelConnector@0.0.0.0:19714, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  12) Thread[id=216, name=qtp453856425-216 Acceptor1 SelectChannelConnector@0.0.0.0:25011, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  13) Thread[id=129, name=qtp390636756-129 Acceptor3 SelectChannelConnector@0.0.0.0:51378, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  14) Thread[id=112, name=qtp1507644908-112 Acceptor2 SelectChannelConnector@0.0.0.0:28779, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  15) Thread[id=176, name=qtp1979690111-176 Acceptor3 SelectChannelConnector@0.0.0.0:15460, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  16) Thread[id=201, name=qtp406322006-201 Acceptor0 SelectChannelConnector@0.0.0.0:30562, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  17) Thread[id=98, name=qtp1431523121-98 Acceptor2 SelectChannelConnector@0.0.0.0:19597, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  18) Thread[id=175, name=qtp1979690111-175 Acceptor2 SelectChannelConnector@0.0.0.0:15460, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  19) Thread[id=113, name=qtp1507644908-113 Acceptor3 SelectChannelConnector@0.0.0.0:28779, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  20) Thread[id=126, name=qtp390636756-126 Acceptor0 SelectChannelConnector@0.0.0.0:51378, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  21) Thread[id=111, name=qtp1507644908-111 Acceptor1 SelectChannelConnector@0.0.0.0:28779, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  22) Thread[id=190, name=qtp109855678-190 Acceptor3 SelectChannelConnector@0.0.0.0:19714, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  23) Thread[id=189, name=qtp109855678-189 Acceptor2 SelectChannelConnector@0.0.0.0:19714, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  24) Thread[id=217, name=qtp453856425-217 Acceptor2 SelectChannelConnector@0.0.0.0:25011, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  25) Thread[id=173, name=qtp1979690111-173 Acceptor0 SelectChannelConnector@0.0.0.0:15460, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  26) Thread[id=152, name=qtp1942734199-152 Acceptor0 SelectChannelConnector@0.0.0.0:10507, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  27) Thread[id=203, name=qtp406322006-203 Acceptor2 SelectChannelConnector@0.0.0.0:30562, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  28) Thread[id=127, name=qtp390636756-127 Acceptor1 SelectChannelConnector@0.0.0.0:51378, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  29) Thread[id=110, name=qtp1507644908-110 Acceptor0 SelectChannelConnector@0.0.0.0:28779, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  30) Thread[id=154, name=qtp1942734199-154 Acceptor2 SelectChannelConnector@0.0.0.0:10507, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  31) Thread[id=96, name=qtp1431523121-96 Acceptor0 SelectChannelConnector@0.0.0.0:19597, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  32) Thread[id=140, name=qtp1432551238-140 Acceptor1 SelectChannelConnector@0.0.0.0:49279, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  33) Thread[id=218, name=qtp453856425-218 Acceptor3 SelectChannelConnector@0.0.0.0:25011, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  34) Thread[id=128, name=qtp390636756-128 Acceptor2 SelectChannelConnector@0.0.0.0:51378, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  35) Thread[id=141, name=qtp1432551238-141 Acceptor2 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
  36) Thread[id=139, name=qtp1432551238-139 Acceptor0 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([8573C2ED46DC4A5E]:0)




Build Log:
[...truncated 8346 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.RecoveryZkTest
[junit4:junit4]   2> 341 T11 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-RecoveryZkTest-1351959534976
[junit4:junit4]   2> 358 T11 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 461 T12 oaz.Environment.logEnv Server environment:zookeeper.version=3.3.6-1366786, built on 07/29/2012 06:22 GMT
[junit4:junit4]   2> 462 T12 oaz.Environment.logEnv Server environment:host.name=lucene.zones.apache.org
[junit4:junit4]   2> 463 T12 oaz.Environment.logEnv Server environment:java.version=1.7.0_09
[junit4:junit4]   2> 463 T12 oaz.Environment.logEnv Server environment:java.vendor=Oracle Corporation
[junit4:junit4]   2> 464 T12 oaz.Environment.logEnv Server environment:java.home=/usr/local/openjdk7/jre
[junit4:junit4]   2> 465 T12 oaz.Environment.logEnv Server environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/common/lucene-analyzers-common-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/highlighter/lucene-highlighter-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/memory/lucene-memory-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/misc/lucene-misc-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/spatial/lucene-spatial-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/suggest/lucene-suggest-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/grouping/lucene-grouping-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queries/lucene-queries-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queryparser/lucene-queryparser-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/easymock-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/guava-r05.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/log4j-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/zookeeper-3.3.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-continuation-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-deploy-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-http-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-io-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-jmx-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-security-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-server-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-servlet-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-util-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-webapp-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-xml-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-javamail.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jdepend.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.4.jar
[junit4:junit4]   2> 470 T12 oaz.Environment.logEnv Server environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
[junit4:junit4]   2> 470 T12 oaz.Environment.logEnv Server environment:java.io.tmpdir=.
[junit4:junit4]   2> 471 T12 oaz.Environment.logEnv Server environment:java.compiler=<NA>
[junit4:junit4]   2> 472 T12 oaz.Environment.logEnv Server environment:os.name=FreeBSD
[junit4:junit4]   2> 472 T12 oaz.Environment.logEnv Server environment:os.arch=amd64
[junit4:junit4]   2> 473 T12 oaz.Environment.logEnv Server environment:os.version=9.0-RELEASE
[junit4:junit4]   2> 473 T12 oaz.Environment.logEnv Server environment:user.name=hudson
[junit4:junit4]   2> 474 T12 oaz.Environment.logEnv Server environment:user.home=/home/hudson
[junit4:junit4]   2> 474 T12 oaz.Environment.logEnv Server environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 518 T12 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 556 T12 oazs.NIOServerCnxn$Factory.<init> binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 568 T11 oasc.ZkTestServer.run start zk server on port:63015
[junit4:junit4]   2> 641 T12 oazsp.FileTxnSnapLog.save Snapshotting: 0
[junit4:junit4]   2> 655 T11 oaz.Environment.logEnv Client environment:zookeeper.version=3.3.6-1366786, built on 07/29/2012 06:22 GMT
[junit4:junit4]   2> 655 T11 oaz.Environment.logEnv Client environment:host.name=lucene.zones.apache.org
[junit4:junit4]   2> 656 T11 oaz.Environment.logEnv Client environment:java.version=1.7.0_09
[junit4:junit4]   2> 657 T11 oaz.Environment.logEnv Client environment:java.vendor=Oracle Corporation
[junit4:junit4]   2> 657 T11 oaz.Environment.logEnv Client environment:java.home=/usr/local/openjdk7/jre
[junit4:junit4]   2> 658 T11 oaz.Environment.logEnv Client environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/common/lucene-analyzers-common-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/highlighter/lucene-highlighter-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/memory/lucene-memory-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/misc/lucene-misc-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/spatial/lucene-spatial-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/suggest/lucene-suggest-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/grouping/lucene-grouping-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queries/lucene-queries-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queryparser/lucene-queryparser-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/easymock-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/guava-r05.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/log4j-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/zookeeper-3.3.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-continuation-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-deploy-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-http-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-io-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-jmx-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-security-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-server-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-servlet-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-util-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-webapp-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-xml-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-javamail.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jdepend.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.4.jar
[junit4:junit4]   2> 661 T11 oaz.Environment.logEnv Client environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
[junit4:junit4]   2> 661 T11 oaz.Environment.logEnv Client environment:java.io.tmpdir=.
[junit4:junit4]   2> 662 T11 oaz.Environment.logEnv Client environment:java.compiler=<NA>
[junit4:junit4]   2> 663 T11 oaz.Environment.logEnv Client environment:os.name=FreeBSD
[junit4:junit4]   2> 663 T11 oaz.Environment.logEnv Client environment:os.arch=amd64
[junit4:junit4]   2> 664 T11 oaz.Environment.logEnv Client environment:os.version=9.0-RELEASE
[junit4:junit4]   2> 665 T11 oaz.Environment.logEnv Client environment:user.name=hudson
[junit4:junit4]   2> 666 T11 oaz.Environment.logEnv Client environment:user.home=/home/hudson
[junit4:junit4]   2> 666 T11 oaz.Environment.logEnv Client environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 671 T11 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63015 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@790d63f8
[junit4:junit4]   2> 731 T15 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:63015
[junit4:junit4]   2> 731 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 736 T15 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63015, initiating session
[junit4:junit4]   2> 737 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:30574
[junit4:junit4]   2> 760 T13 oazs.NIOServerCnxn.doIO WARNING Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
[junit4:junit4]   2> 761 T13 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:30574 (no session established for client)
[junit4:junit4]   2> 762 T15 oaz.ClientCnxn$SendThread.run Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
[junit4:junit4]   2> 2187 T15 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost/127.0.0.1:63015
[junit4:junit4]   2> 2188 T15 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63015, initiating session
[junit4:junit4]   2> 2189 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:26698
[junit4:junit4]   2> 2191 T13 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:26698
[junit4:junit4]   2> 2208 T16 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 2245 T16 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac7115ecc0000 with negotiated timeout 10000 for client /140.211.11.196:26698
[junit4:junit4]   2> 2246 T15 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:63015, sessionid = 0x13ac7115ecc0000, negotiated timeout = 10000
[junit4:junit4]   2> 2278 T17 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@790d63f8 name:ZooKeeperConnection Watcher:127.0.0.1:63015 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2279 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2285 T11 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 2389 T18 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x13ac7115ecc0000
[junit4:junit4]   2> 2392 T17 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 2391 T11 oaz.ZooKeeper.close Session: 0x13ac7115ecc0000 closed
[junit4:junit4]   2> 2393 T13 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:26698 which had sessionid 0x13ac7115ecc0000
[junit4:junit4]   2> 2393 T11 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63015/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@5702cbd3
[junit4:junit4]   2> 2396 T19 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:63015
[junit4:junit4]   2> 2396 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2397 T19 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63015, initiating session
[junit4:junit4]   2> 2398 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:10099
[junit4:junit4]   2> 2399 T13 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:10099
[junit4:junit4]   2> 2403 T16 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac7115ecc0001 with negotiated timeout 10000 for client /140.211.11.196:10099
[junit4:junit4]   2> 2403 T19 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:63015, sessionid = 0x13ac7115ecc0001, negotiated timeout = 10000
[junit4:junit4]   2> 2404 T20 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5702cbd3 name:ZooKeeperConnection Watcher:127.0.0.1:63015/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2405 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2427 T11 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 2435 T11 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 2441 T11 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 2446 T11 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 2452 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2523 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2540 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 2542 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 2549 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2550 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2572 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 2573 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 2579 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 2580 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 2585 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 2586 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 2591 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 2592 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 2597 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 2599 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 2604 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 2605 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 2610 T11 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 2611 T11 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 2616 T18 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x13ac7115ecc0001
[junit4:junit4]   2> 2618 T20 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 2618 T11 oaz.ZooKeeper.close Session: 0x13ac7115ecc0001 closed
[junit4:junit4]   2> 2619 T13 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:10099 which had sessionid 0x13ac7115ecc0001
[junit4:junit4]   2> 3083 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 3380 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:19540
[junit4:junit4]   2> 3667 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3669 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 3670 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4662 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 4668 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4668 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 4713 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 4719 T11 oasc.CoreContainer.<init> New CoreContainer 1069263533
[junit4:junit4]   2> 4720 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4721 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4829 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4864 T11 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:63015/solr
[junit4:junit4]   2> 4920 T11 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63015/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@7659657a
[junit4:junit4]   2> 4921 T35 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:63015
[junit4:junit4]   2> 4921 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4923 T35 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63015, initiating session
[junit4:junit4]   2> 4924 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:31390
[junit4:junit4]   2> 4926 T13 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:31390
[junit4:junit4]   2> 4929 T16 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac7115ecc0002 with negotiated timeout 5000 for client /140.211.11.196:31390
[junit4:junit4]   2> 4929 T35 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:63015, sessionid = 0x13ac7115ecc0002, negotiated timeout = 5000
[junit4:junit4]   2> 4930 T36 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7659657a name:ZooKeeperConnection Watcher:127.0.0.1:63015/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4930 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4953 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0002 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4966 T11 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 4970 T11 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:19540_solr
[junit4:junit4]   2> 4977 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0002 type:delete cxid:0x7 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:19540_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:19540_solr
[junit4:junit4]   2> 4978 T11 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:19540_solr
[junit4:junit4]   2> 5004 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 5005 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 5005 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 5006 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 5006 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 5007 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 5008 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 5008 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 5009 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 5081 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 5723 T11 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 5762 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0002 type:delete cxid:0x15 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 5764 T11 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 5768 T11 oasc.Overseer.start Overseer (id=88602020105945090-127.0.0.1:19540_solr-n_0000000000) starting
[junit4:junit4]   2> 5769 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0002 type:create cxid:0x1a zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5785 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0002 type:create cxid:0x1b zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5787 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0002 type:create cxid:0x1c zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5793 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0002 type:create cxid:0x1d zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5795 T38 oasc.OverseerCollectionProcessor.run Process current queue of collection messages
[junit4:junit4]   2> 5796 T11 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 5800 T11 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 5824 T37 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 5889 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 5890 T11 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 5891 T11 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 5892 T11 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 5894 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 5896 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 5898 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 6135 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 6395 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 6425 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 6500 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 8633 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 8668 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 8673 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 9058 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9081 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9088 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9097 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9189 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1351959534975/control/data/
[junit4:junit4]   2> 9196 T11 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@12f794ce
[junit4:junit4]   2> 9458 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1351959534975/control/data/index/
[junit4:junit4]   2> 9460 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1351959534975/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 9573 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-1351959534975/control/data/index forceNew:false
[junit4:junit4]   2> 10084 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1351959534975/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@46f0474a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 10085 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 10118 T11 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 11080 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 11082 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 11082 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 11092 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 11232 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 11247 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 11253 T11 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 11290 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 11310 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 11359 T11 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 11361 T11 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 11367 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 11368 T11 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 11369 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 11369 T11 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 11398 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 11399 T11 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11400 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11401 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11403 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11404 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11405 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11406 T11 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11426 T11 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 11445 T11 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 11574 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 11608 T11 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 11617 T11 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 11677 T11 oass.SolrIndexSearcher.<init> Opening Searcher@38219f32 main
[junit4:junit4]   2> 11765 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 11765 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 11767 T11 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 11873 T11 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 12273 T39 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@38219f32 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 12291 T11 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 12293 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0002 type:create cxid:0x74 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 12306 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 12307 T11 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:19540/solr shardId:control_shard
[junit4:junit4]   2> 12324 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 12341 T11 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/control_shard/election
[junit4:junit4]   2> 12359 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0002 type:delete cxid:0x84 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 12366 T11 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 12368 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0002 type:create cxid:0x88 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12368 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12370 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"control_shard",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:19540_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:19540/solr"}
[junit4:junit4]   2> 12386 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0002 type:create cxid:0x8b zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 12426 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (1)
[junit4:junit4]   2> 12873 T11 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 12874 T11 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 12875 T11 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:19540/solr/collection1/
[junit4:junit4]   2> 12876 T11 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 12876 T11 oasc.SyncStrategy.syncToMe http://127.0.0.1:19540/solr/collection1/ has no replicas
[junit4:junit4]   2> 12877 T11 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:19540/solr/collection1/
[junit4:junit4]   2> 12878 T11 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/control_shard
[junit4:junit4]   2> 12885 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0002 type:create cxid:0xa9 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12935 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12949 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (1)
[junit4:junit4]   2> 12998 T11 oasc.ZkController.register We are http://127.0.0.1:19540/solr/collection1/ and leader is http://127.0.0.1:19540/solr/collection1/
[junit4:junit4]   2> 13003 T11 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:19540/solr
[junit4:junit4]   2> 13003 T11 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 13004 T11 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 13009 T11 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13010 T11 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 13011 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 13088 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 13303 T11 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 13306 T11 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:34199
[junit4:junit4]   2> 13308 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 13308 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351959547740
[junit4:junit4]   2> 13309 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351959547740/'
[junit4:junit4]   2> 13360 T11 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 13361 T11 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 13361 T11 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351959547740
[junit4:junit4]   2> 13362 T11 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351959547740/solr.xml
[junit4:junit4]   2> 13363 T11 oasc.CoreContainer.<init> New CoreContainer 1693106741
[junit4:junit4]   2> 13364 T11 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351959547740/'
[junit4:junit4]   2> 13365 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351959547740/'
[junit4:junit4]   2> 13407 T11 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 13429 T11 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:63015/solr
[junit4:junit4]   2> 13430 T11 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:63015/solr sessionTimeout=5000 watcher=org.apache.solr.common.cloud.ConnectionManager@768db0c6
[junit4:junit4]   2> 13432 T49 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:63015
[junit4:junit4]   2> 13432 T11 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 13434 T49 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:63015, initiating session
[junit4:junit4]   2> 13434 T13 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:57041
[junit4:junit4]   2> 13435 T13 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:57041
[junit4:junit4]   2> 13518 T16 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac7115ecc0003 with negotiated timeout 5000 for client /140.211.11.196:57041
[junit4:junit4]   2> 13518 T49 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:63015, sessionid = 0x13ac7115ecc0003, negotiated timeout = 5000
[junit4:junit4]   2> 13520 T50 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@768db0c6 name:ZooKeeperConnection Watcher:127.0.0.1:63015/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 13520 T11 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 13521 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13522 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0003 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13524 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0003 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13525 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"control_shard",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:19540_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:19540/solr"}
[junit4:junit4]   2> 13536 T11 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34199_solr
[junit4:junit4]   2> 13537 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0003 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:34199_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:34199_solr
[junit4:junit4]   2> 13539 T11 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34199_solr
[junit4:junit4]   2> 13542 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (1)
[junit4:junit4]   2> 13546 T11 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 13547 T11 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 13548 T11 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 13548 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 13549 T36 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 13549 T11 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 13550 T11 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 13550 T11 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 13551 T11 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 13551 T11 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 13552 T11 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 13577 T11 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 13585 T11 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351959547740/collection1
[junit4:junit4]   2> 13586 T11 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 13587 T11 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 13588 T11 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 13590 T11 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351959547740/collection1/'
[junit4:junit4]   2> 13592 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-jetty1-1351959547740/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 13593 T11 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-jetty1-1351959547740/collection1/lib/README' to classloader
[junit4:junit4]   2> 13687 T11 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 13774 T11 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 13776 T11 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 13822 T11 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 14489 T11 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 14504 T11 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 14509 T11 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 14537 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 14544 T11 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 14550 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 14554 T11 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 14559 T11 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-jetty1-1351959547740/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1351959534975/jetty1/
[junit4:junit4]   2> 14559 T11 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@12f794ce
[junit4:junit4]   2> 14561 T11 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1351959534975/jetty1/index/
[junit4:junit4]   2> 14562 T11 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1351959534975/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 14563 T11 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.RecoveryZkTest-1351959534975/jetty1/index forceNew:false
[junit4:junit4]   2> 14571 T11 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.RecoveryZkTest-1351959534975/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@82d195a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 14572 T11 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 14574 T11 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 14579 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 14580 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 14581 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 14582 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 14583 T11 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 14584 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 14585 T11 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 14587 T11 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 14589 T11 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 14590 T11 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 14592 T11 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 14593 T11 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 14594 T11 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 14595 T11 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 14596 T11 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 14597 T11 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 14599 T11 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14600 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14602 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14603 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14605 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14606 T11 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14609 T11 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14612 T11 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 14617 T11 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 14637 T11 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 14642 T11 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 14644 T11 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 14648 T11 oass.SolrIndexSearcher.<init> Opening Searcher@6b861ad2 main
[junit4:junit4]   2> 14650 T11 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 14650 T11 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 14652 T11 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 14682 T11 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 14853 T55 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6b861ad2 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 14865 T11 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15054 T37 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15055 T37 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:34199_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:34199/solr"}
[junit4:junit4]   2> 15055 T37 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4]   2> 15084 T50 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 15084 T36 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 15389 T11 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 15389 T11 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:34199/solr shardId:shard1
[junit4:junit4]   2> 15391 T11 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 15411 T18 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac7115ecc0003 type:delete cxid:0x42 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr

[...truncated too long message...]

se.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   27) Thread[id=203, name=qtp406322006-203 Acceptor2 SelectChannelConnector@0.0.0.0:30562, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   28) Thread[id=127, name=qtp390636756-127 Acceptor1 SelectChannelConnector@0.0.0.0:51378, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   29) Thread[id=110, name=qtp1507644908-110 Acceptor0 SelectChannelConnector@0.0.0.0:28779, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   30) Thread[id=154, name=qtp1942734199-154 Acceptor2 SelectChannelConnector@0.0.0.0:10507, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   31) Thread[id=96, name=qtp1431523121-96 Acceptor0 SelectChannelConnector@0.0.0.0:19597, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   32) Thread[id=140, name=qtp1432551238-140 Acceptor1 SelectChannelConnector@0.0.0.0:49279, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   33) Thread[id=218, name=qtp453856425-218 Acceptor3 SelectChannelConnector@0.0.0.0:25011, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   34) Thread[id=128, name=qtp390636756-128 Acceptor2 SelectChannelConnector@0.0.0.0:51378, state=RUNNABLE, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   35) Thread[id=141, name=qtp1432551238-141 Acceptor2 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >   36) Thread[id=139, name=qtp1432551238-139 Acceptor0 SelectChannelConnector@0.0.0.0:49279, state=BLOCKED, group=TGRP-DistributedSpellCheckComponentTest]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([8573C2ED46DC4A5E]:0)
[junit4:junit4] Completed on J0 in 40.15s, 1 test, 2 errors <<< FAILURES!

[...truncated 473 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:335: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/module-build.xml:61: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:1163: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:827: There were test failures: 233 suites, 965 tests, 4 suite-level errors, 825 ignored

Total time: 31 minutes 10 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 609 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/609/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.LeaderElectionIntegrationTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([74596681D419C368]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.request.TestRemoteStreaming

Error Message:
4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming:     1) Thread[id=35, name=qtp1053008442-35 Acceptor0 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=37, name=qtp1053008442-37 Acceptor2 SelectChannelConnector@0.0.0.0:50599, state=RUNNABLE, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=38, name=qtp1053008442-38 Acceptor3 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=36, name=qtp1053008442-36 Acceptor1 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming: 
   1) Thread[id=35, name=qtp1053008442-35 Acceptor0 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=37, name=qtp1053008442-37 Acceptor2 SelectChannelConnector@0.0.0.0:50599, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=38, name=qtp1053008442-38 Acceptor3 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=36, name=qtp1053008442-36 Acceptor1 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([74596681D419C368]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.request.TestRemoteStreaming

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=35, name=qtp1053008442-35 Acceptor0 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    2) Thread[id=37, name=qtp1053008442-37 Acceptor2 SelectChannelConnector@0.0.0.0:50599, state=RUNNABLE, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    3) Thread[id=38, name=qtp1053008442-38 Acceptor3 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)    4) Thread[id=36, name=qtp1053008442-36 Acceptor1 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)         at java.lang.Thread.run(Thread.java:722)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=35, name=qtp1053008442-35 Acceptor0 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   2) Thread[id=37, name=qtp1053008442-37 Acceptor2 SelectChannelConnector@0.0.0.0:50599, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   3) Thread[id=38, name=qtp1053008442-38 Acceptor3 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
   4) Thread[id=36, name=qtp1053008442-36 Acceptor1 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
        at java.lang.Thread.run(Thread.java:722)
	at __randomizedtesting.SeedInfo.seed([74596681D419C368]:0)


FAILED:  org.apache.solr.cloud.LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([74596681D419C368]:0)




Build Log:
[...truncated 8297 lines...]
[junit4:junit4] Suite: org.apache.solr.request.TestRemoteStreaming
[junit4:junit4]   2> 0 T28 oas.SolrTestCaseJ4.initCore ####initCore
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/./solrtest-TestRemoteStreaming-1351906047269
[junit4:junit4]   2> 2 T28 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4:junit4]   2> 573 T28 oejs.Server.doStart jetty-8.1.2.v20120308
[junit4:junit4]   2> 917 T28 oejs.AbstractConnector.doStart Started SelectChannelConnector@0.0.0.0:50599
[junit4:junit4]   2> 959 T28 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 960 T28 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: solr/
[junit4:junit4]   2> 960 T28 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: 'solr/'
[junit4:junit4]   2> 1004 T28 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1005 T28 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1005 T28 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: solr/
[junit4:junit4]   2> 1006 T28 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solr/solr.xml
[junit4:junit4]   2> 1006 T28 oasc.CoreContainer.<init> New CoreContainer 1645340219
[junit4:junit4]   2> 1007 T28 oasc.CoreContainer$Initializer.initialize no solr.xml file found - using default
[junit4:junit4]   2> 1008 T28 oasc.CoreContainer.load Loading CoreContainer using Solr Home: 'solr/'
[junit4:junit4]   2> 1008 T28 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'solr/'
[junit4:junit4]   2> 1053 T28 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 1109 T28 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: solr/collection1
[junit4:junit4]   2> 1110 T28 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'solr/collection1/'
[junit4:junit4]   2> 1186 T28 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 1257 T28 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1261 T28 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 1290 T28 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1883 T28 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1896 T28 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1900 T28 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1943 T28 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1967 T28 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1974 T28 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1982 T28 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1986 T28 oasc.SolrCore.<init> [collection1] Opening new SolrCore at solr/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-TestRemoteStreaming-1351906047269/
[junit4:junit4]   2> 1986 T28 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2bf96987
[junit4:junit4]   2> 1989 T28 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-TestRemoteStreaming-1351906047269/index/
[junit4:junit4]   2> 1990 T28 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-TestRemoteStreaming-1351906047269/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1992 T28 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1/solrtest-TestRemoteStreaming-1351906047269/index forceNew:false
[junit4:junit4]   2> 1996 T28 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4bc2ebc7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3c3695),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1997 T28 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 1999 T28 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 2004 T28 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2006 T28 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 2007 T28 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2008 T28 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 2010 T28 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2011 T28 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2012 T28 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2014 T28 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2015 T28 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2017 T28 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 2019 T28 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 2020 T28 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2021 T28 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 2022 T28 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 2024 T28 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 2025 T28 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2027 T28 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2028 T28 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2030 T28 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2032 T28 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2033 T28 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2035 T28 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2036 T28 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2038 T28 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 2039 T28 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 2063 T28 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2072 T28 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 2075 T28 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: []
[junit4:junit4]   2> 2080 T28 oass.SolrIndexSearcher.<init> Opening Searcher@33a40908 main
[junit4:junit4]   2> 2081 T28 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 2083 T28 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2084 T28 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2085 T28 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 2129 T28 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 2361 T28 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 2362 T28 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 2363 T28 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 2364 T28 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 2364 T28 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 2365 T28 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 2365 T28 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 2366 T28 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 2367 T28 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 2367 T28 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 2391 T43 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@33a40908 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2405 T28 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 2406 T28 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 2406 T28 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 2407 T28 oas.SolrJettyTestBase.createJetty Jetty Assigned Port#50599
[junit4:junit4]   2> 2424 T28 oas.SolrTestCaseJ4.setUp ###Starting testNoUrlAccess
[junit4:junit4]   2> 2526 T28 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C5 name=collection1 org.apache.solr.core.SolrCore@4d8267cc
[junit4:junit4]   2> 3774 T40 C5 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4bc2ebc7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3c3695),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 3774 T40 C5 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 3776 T40 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1234]} 0 33
[junit4:junit4]   2> 3910 T41 C5 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 3948 T41 C5 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4bc2ebc7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3c3695),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4bc2ebc7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3c3695),segFN=segments_2,generation=2,filenames=[_0.fnm, _0_Memory_0.ram, _0_MockVariableIntBlock_0.doc, _0_nrm.cfs, _0_MockVariableIntBlock_0.skp, _0_Memory_1.ram, _0_MockVariableIntBlock_0.pyl, _0_Pulsing41_0.doc, _0_Pulsing41_0.pos, _0_MockVariableIntBlock_0.tib, _0_Pulsing41_0.tim, _0_Pulsing41_0.tip, _0_MockVariableIntBlock_0.frq, _0_nrm.cfe, _0_MockVariableIntBlock_0.pos, segments_2, _0.fdx, _0.si, _0.fdt, _0_MockVariableIntBlock_0.tii]
[junit4:junit4]   2> 3948 T41 C5 oasc.SolrDeletionPolicy.updateCommits newest commit = 2
[junit4:junit4]   2> 3966 T41 C5 oass.SolrIndexSearcher.<init> Opening Searcher@4f8771a main
[junit4:junit4]   2> 3966 T41 C5 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 3967 T41 C5 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 3969 T43 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4f8771a main{StandardDirectoryReader(segments_2:3 _0(4.1):C1)}
[junit4:junit4]   2> 3970 T41 C5 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 60
[junit4:junit4]   2> 4051 T42 C5 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=56 
[junit4:junit4]   2> 4062 T41 C5 REQ [collection1] webapp=/solr path=/select params={wt=javabin&stream.url=http://127.0.0.1:50599/solr/update?commit%3Dtrue%26stream.body%3D%253Cdelete%253E%253Cquery%253E*%253A*%253C%252Fquery%253E%253C%252Fdelete%253E&q=*:*&version=2} hits=1 status=0 QTime=5 
[junit4:junit4]   2> 4066 T42 C5 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 4067 T28 oas.SolrTestCaseJ4.tearDown ###Ending testNoUrlAccess
[junit4:junit4]   2> 4076 T28 oas.SolrTestCaseJ4.setUp ###Starting testStreamUrl
[junit4:junit4]   2> 4080 T40 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1234]} 0 1
[junit4:junit4]   2> 4084 T41 C5 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 4089 T41 C5 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4bc2ebc7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3c3695),segFN=segments_2,generation=2,filenames=[_0.fnm, _0_Memory_0.ram, _0_MockVariableIntBlock_0.doc, _0_nrm.cfs, _0_MockVariableIntBlock_0.skp, _0_Memory_1.ram, _0_MockVariableIntBlock_0.pyl, _0_Pulsing41_0.doc, _0_Pulsing41_0.pos, _0_MockVariableIntBlock_0.tib, _0_Pulsing41_0.tim, _0_Pulsing41_0.tip, _0_MockVariableIntBlock_0.frq, _0_nrm.cfe, _0_MockVariableIntBlock_0.pos, segments_2, _0.fdx, _0.si, _0.fdt, _0_MockVariableIntBlock_0.tii]
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4bc2ebc7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3c3695),segFN=segments_3,generation=3,filenames=[_1_MockVariableIntBlock_0.skp, _1_nrm.cfs, _1_Memory_1.ram, _1_MockVariableIntBlock_0.tii, _1_MockVariableIntBlock_0.pos, _1_MockVariableIntBlock_0.pyl, _1_Pulsing41_0.doc, _1.fnm, _1_Pulsing41_0.pos, _1_nrm.cfe, _1_MockVariableIntBlock_0.tib, _1_MockVariableIntBlock_0.frq, _1.fdx, _1_Memory_0.ram, _1_Pulsing41_0.tip, _1.fdt, _1_Pulsing41_0.tim, _1.si, _1_MockVariableIntBlock_0.doc, segments_3]
[junit4:junit4]   2> 4090 T41 C5 oasc.SolrDeletionPolicy.updateCommits newest commit = 3
[junit4:junit4]   2> 4092 T41 C5 oass.SolrIndexSearcher.<init> Opening Searcher@11ad05de main
[junit4:junit4]   2> 4093 T41 C5 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 4093 T41 C5 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 4096 T43 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@11ad05de main{StandardDirectoryReader(segments_3:7 _1(4.1):C1)}
[junit4:junit4]   2> 4097 T41 C5 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 13
[junit4:junit4]   2> 4105 T42 C5 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 4121 T42 C5 REQ [collection1] webapp=/solr path=/select params={wt=csv&q=*:*&fl=id} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 4148 T41 C5 REQ [collection1] webapp=/solr path=/debug/dump params={wt=xml&stream.url=http://127.0.0.1:50599/solr/select?q%3D*:*%26fl%3Did%26wt%3Dcsv} status=0 QTime=30 
[junit4:junit4]   2> 4157 T28 oas.SolrTestCaseJ4.tearDown ###Ending testStreamUrl
[junit4:junit4]   2> 4166 T28 oas.SolrTestCaseJ4.setUp ###Starting testMakeDeleteAllUrl
[junit4:junit4]   2> 4171 T40 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1234]} 0 1
[junit4:junit4]   2> 4175 T41 C5 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 4180 T41 C5 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4bc2ebc7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3c3695),segFN=segments_3,generation=3,filenames=[_1_MockVariableIntBlock_0.skp, _1_nrm.cfs, _1_Memory_1.ram, _1_MockVariableIntBlock_0.tii, _1_MockVariableIntBlock_0.pos, _1_MockVariableIntBlock_0.pyl, _1_Pulsing41_0.doc, _1.fnm, _1_Pulsing41_0.pos, _1_nrm.cfe, _1_MockVariableIntBlock_0.tib, _1_MockVariableIntBlock_0.frq, _1.fdx, _1_Memory_0.ram, _1_Pulsing41_0.tip, _1.fdt, _1_Pulsing41_0.tim, _1.si, _1_MockVariableIntBlock_0.doc, segments_3]
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4bc2ebc7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3c3695),segFN=segments_4,generation=4,filenames=[_2_Pulsing41_0.doc, _2.si, _2_MockVariableIntBlock_0.tii, _2_MockVariableIntBlock_0.pyl, _2_MockVariableIntBlock_0.tib, _2.fdt, _2_MockVariableIntBlock_0.frq, _2.fdx, _2_MockVariableIntBlock_0.doc, _2.fnm, _2_Pulsing41_0.tim, _2_Pulsing41_0.pos, _2_nrm.cfe, _2_Memory_1.ram, _2_Pulsing41_0.tip, _2_Memory_0.ram, _2_MockVariableIntBlock_0.skp, _2_nrm.cfs, _2_MockVariableIntBlock_0.pos, segments_4]
[junit4:junit4]   2> 4181 T41 C5 oasc.SolrDeletionPolicy.updateCommits newest commit = 4
[junit4:junit4]   2> 4183 T41 C5 oass.SolrIndexSearcher.<init> Opening Searcher@24fd5136 main
[junit4:junit4]   2> 4184 T41 C5 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 4184 T41 C5 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 4187 T43 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@24fd5136 main{StandardDirectoryReader(segments_4:11 _2(4.1):C1)}
[junit4:junit4]   2> 4188 T41 C5 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 13
[junit4:junit4]   2> 4192 T42 C5 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 4195 T40 C5 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4:junit4]   2> 4196 T40 C5 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 4197 T40 C5 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4bc2ebc7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3c3695),segFN=segments_4,generation=4,filenames=[_2_Pulsing41_0.doc, _2.si, _2_MockVariableIntBlock_0.tii, _2_MockVariableIntBlock_0.pyl, _2_MockVariableIntBlock_0.tib, _2.fdt, _2_MockVariableIntBlock_0.frq, _2.fdx, _2_MockVariableIntBlock_0.doc, _2.fnm, _2_Pulsing41_0.tim, _2_Pulsing41_0.pos, _2_nrm.cfe, _2_Memory_1.ram, _2_Pulsing41_0.tip, _2_Memory_0.ram, _2_MockVariableIntBlock_0.skp, _2_nrm.cfs, _2_MockVariableIntBlock_0.pos, segments_4]
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4bc2ebc7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3c3695),segFN=segments_5,generation=5,filenames=[segments_5]
[junit4:junit4]   2> 4197 T40 C5 oasc.SolrDeletionPolicy.updateCommits newest commit = 5
[junit4:junit4]   2> 4198 T40 C5 oass.SolrIndexSearcher.<init> Opening Searcher@782a437c main
[junit4:junit4]   2> 4199 T40 C5 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 4199 T40 C5 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 4201 T43 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@782a437c main{StandardDirectoryReader(segments_5:12)}
[junit4:junit4]   2> 4202 T40 C5 UPDATE [collection1] webapp=/solr path=/update params={commit=true&stream.body=<delete><query>*:*</query></delete>} {deleteByQuery=*:*,commit=} 0 8
[junit4:junit4]   2> 4207 T41 C5 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=0 status=0 QTime=1 
[junit4:junit4]   2> 4208 T28 oas.SolrTestCaseJ4.tearDown ###Ending testMakeDeleteAllUrl
[junit4:junit4]   2> 4216 T28 oas.SolrTestCaseJ4.setUp ###Starting testQtUpdateFails
[junit4:junit4]   2> 4222 T42 C5 UPDATE [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1234]} 0 2
[junit4:junit4]   2> 4227 T40 C5 oasu.DirectUpdateHandler2.commit start commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 4233 T40 C5 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4bc2ebc7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3c3695),segFN=segments_5,generation=5,filenames=[segments_5]
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4bc2ebc7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3c3695),segFN=segments_6,generation=6,filenames=[_3_nrm.cfe, _3_MockVariableIntBlock_0.frq, _3_MockVariableIntBlock_0.doc, _3.fnm, _3.fdt, _3_MockVariableIntBlock_0.tii, _3.fdx, _3_Pulsing41_0.pos, _3_nrm.cfs, _3_MockVariableIntBlock_0.pyl, _3_MockVariableIntBlock_0.tib, _3_Pulsing41_0.tim, _3.si, _3_Pulsing41_0.tip, _3_Memory_0.ram, _3_MockVariableIntBlock_0.skp, segments_6, _3_Pulsing41_0.doc, _3_Memory_1.ram, _3_MockVariableIntBlock_0.pos]
[junit4:junit4]   2> 4233 T40 C5 oasc.SolrDeletionPolicy.updateCommits newest commit = 6
[junit4:junit4]   2> 4237 T40 C5 oass.SolrIndexSearcher.<init> Opening Searcher@74e6cbcd main
[junit4:junit4]   2> 4237 T40 C5 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 4238 T40 C5 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 4241 T43 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@74e6cbcd main{StandardDirectoryReader(segments_6:14 _3(4.1):C1)}
[junit4:junit4]   2> 4241 T40 C5 UPDATE [collection1] webapp=/solr path=/update params={waitSearcher=true&wt=javabin&commit=true&softCommit=false&version=2} {commit=} 0 14
[junit4:junit4]   2> 4246 T41 C5 REQ [collection1] webapp=/solr path=/select params={wt=javabin&q=id:1234&version=2} hits=1 status=0 QTime=1 
[junit4:junit4]   2> 4250 T28 oas.SolrTestCaseJ4.tearDown ###Ending testQtUpdateFails
[junit4:junit4]   2> 4298 T28 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1645340219
[junit4:junit4]   2> 4298 T28 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@4d8267cc
[junit4:junit4]   2> 4313 T28 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=5,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=4,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4]   2> 4314 T28 oasc.SolrCore.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 4314 T28 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 4315 T28 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 4405 T28 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 4417 T28 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4]   2> 4518 T28 oejut.QueuedThreadPool.doStop WARNING 4 threads could not be stopped
[junit4:junit4]   2> 4518 T28 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> 4532 T27 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 4 leaked thread(s).
[junit4:junit4]   2> 24664 T27 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming: 
[junit4:junit4]   2> 	   1) Thread[id=35, name=qtp1053008442-35 Acceptor0 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   2) Thread[id=37, name=qtp1053008442-37 Acceptor2 SelectChannelConnector@0.0.0.0:50599, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   3) Thread[id=38, name=qtp1053008442-38 Acceptor3 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   4) Thread[id=36, name=qtp1053008442-36 Acceptor1 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 24665 T27 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
[junit4:junit4]   2> 	   1) Thread[id=35, name=qtp1053008442-35 Acceptor0 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	   2) Thread[id=37, name=qtp1053008442-37 Acceptor2 SelectChannelConnector@0.0.0.0:50599, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	   3) Thread[id=38, name=qtp1053008442-38 Acceptor3 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	   4) Thread[id=36, name=qtp1053008442-36 Acceptor1 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 27708 T27 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
[junit4:junit4]   2> 	   1) Thread[id=35, name=qtp1053008442-35 Acceptor0 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   2) Thread[id=37, name=qtp1053008442-37 Acceptor2 SelectChannelConnector@0.0.0.0:50599, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   3) Thread[id=38, name=qtp1053008442-38 Acceptor3 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	   4) Thread[id=36, name=qtp1053008442-36 Acceptor1 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]   2> 	        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene41: {timestamp=Pulsing41(freqCutoff=16 minBlockSize=6 maxBlockSize=102), range_facet_l=Pulsing41(freqCutoff=16 minBlockSize=6 maxBlockSize=102), id=PostingsFormat(name=Memory doPackFST= false), range_facet_sl=PostingsFormat(name=Memory doPackFST= true), range_facet_si=MockVariableIntBlock(baseBlockSize=86), multiDefault=MockVariableIntBlock(baseBlockSize=86), intDefault=Pulsing41(freqCutoff=16 minBlockSize=6 maxBlockSize=102)}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=fi, timezone=Africa/Algiers
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=5,free=38914040,total=145752064
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestCharFilters, SoftAutoCommitTest, TermVectorComponentTest, TestRemoteStreaming]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestRemoteStreaming -Dtests.seed=74596681D419C368 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=fi -Dtests.timezone=Africa/Algiers -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] ERROR   0.00s J1 | TestRemoteStreaming (suite) <<<
[junit4:junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.request.TestRemoteStreaming: 
[junit4:junit4]    >    1) Thread[id=35, name=qtp1053008442-35 Acceptor0 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=37, name=qtp1053008442-37 Acceptor2 SelectChannelConnector@0.0.0.0:50599, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=38, name=qtp1053008442-38 Acceptor3 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=36, name=qtp1053008442-36 Acceptor1 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([74596681D419C368]:0)
[junit4:junit4]    > Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4]    >    1) Thread[id=35, name=qtp1053008442-35 Acceptor0 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    2) Thread[id=37, name=qtp1053008442-37 Acceptor2 SelectChannelConnector@0.0.0.0:50599, state=RUNNABLE, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    3) Thread[id=38, name=qtp1053008442-38 Acceptor3 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    >    4) Thread[id=36, name=qtp1053008442-36 Acceptor1 SelectChannelConnector@0.0.0.0:50599, state=BLOCKED, group=TGRP-TestRemoteStreaming]
[junit4:junit4]    >         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:210)
[junit4:junit4]    >         at org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:104)
[junit4:junit4]    >         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:933)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
[junit4:junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
[junit4:junit4]    >         at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([74596681D419C368]:0)
[junit4:junit4] Completed on J1 in 27.92s, 4 tests, 2 errors <<< FAILURES!

[...truncated 639 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.LeaderElectionIntegrationTest
[junit4:junit4]   2> 4 T75 oas.SolrTestCaseJ4.setUp ###Starting testLeaderElectionAfterClientTimeout
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841
[junit4:junit4]   2> 10 T75 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 133 T76 oaz.Environment.logEnv Server environment:zookeeper.version=3.3.6-1366786, built on 07/29/2012 06:22 GMT
[junit4:junit4]   2> 134 T76 oaz.Environment.logEnv Server environment:host.name=lucene.zones.apache.org
[junit4:junit4]   2> 134 T76 oaz.Environment.logEnv Server environment:java.version=1.7.0_09
[junit4:junit4]   2> 134 T76 oaz.Environment.logEnv Server environment:java.vendor=Oracle Corporation
[junit4:junit4]   2> 134 T76 oaz.Environment.logEnv Server environment:java.home=/usr/local/openjdk7/jre
[junit4:junit4]   2> 134 T76 oaz.Environment.logEnv Server environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/common/lucene-analyzers-common-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/highlighter/lucene-highlighter-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/memory/lucene-memory-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/misc/lucene-misc-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/spatial/lucene-spatial-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/suggest/lucene-suggest-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/grouping/lucene-grouping-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queries/lucene-queries-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queryparser/lucene-queryparser-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/easymock-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/guava-r05.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/log4j-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/zookeeper-3.3.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-continuation-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-deploy-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-http-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-io-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-jmx-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-security-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-server-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-servlet-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-util-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-webapp-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-xml-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-javamail.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jdepend.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.4.jar
[junit4:junit4]   2> 135 T76 oaz.Environment.logEnv Server environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
[junit4:junit4]   2> 136 T76 oaz.Environment.logEnv Server environment:java.io.tmpdir=.
[junit4:junit4]   2> 136 T76 oaz.Environment.logEnv Server environment:java.compiler=<NA>
[junit4:junit4]   2> 136 T76 oaz.Environment.logEnv Server environment:os.name=FreeBSD
[junit4:junit4]   2> 136 T76 oaz.Environment.logEnv Server environment:os.arch=amd64
[junit4:junit4]   2> 136 T76 oaz.Environment.logEnv Server environment:os.version=9.0-RELEASE
[junit4:junit4]   2> 137 T76 oaz.Environment.logEnv Server environment:user.name=hudson
[junit4:junit4]   2> 137 T76 oaz.Environment.logEnv Server environment:user.home=/home/hudson
[junit4:junit4]   2> 137 T76 oaz.Environment.logEnv Server environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 189 T76 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 233 T76 oazs.NIOServerCnxn$Factory.<init> binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 315 T76 oazsp.FileTxnSnapLog.save Snapshotting: 0
[junit4:junit4]   2> 318 T75 oasc.ZkTestServer.run start zk server on port:57157
[junit4:junit4]   2> 405 T75 oaz.Environment.logEnv Client environment:zookeeper.version=3.3.6-1366786, built on 07/29/2012 06:22 GMT
[junit4:junit4]   2> 406 T75 oaz.Environment.logEnv Client environment:host.name=lucene.zones.apache.org
[junit4:junit4]   2> 406 T75 oaz.Environment.logEnv Client environment:java.version=1.7.0_09
[junit4:junit4]   2> 406 T75 oaz.Environment.logEnv Client environment:java.vendor=Oracle Corporation
[junit4:junit4]   2> 406 T75 oaz.Environment.logEnv Client environment:java.home=/usr/local/openjdk7/jre
[junit4:junit4]   2> 407 T75 oaz.Environment.logEnv Client environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/common/lucene-analyzers-common-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/highlighter/lucene-highlighter-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/memory/lucene-memory-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/misc/lucene-misc-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/spatial/lucene-spatial-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/suggest/lucene-suggest-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/grouping/lucene-grouping-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queries/lucene-queries-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/queryparser/lucene-queryparser-4.1-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/easymock-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/guava-r05.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/log4j-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/solrj/lib/zookeeper-3.3.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-continuation-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-deploy-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-http-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-io-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-jmx-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-security-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-server-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-servlet-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-util-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-webapp-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/jetty-xml-8.1.2.v20120308.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-javamail.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/Ant/ant-1.8.2/lib/ant-jdepend.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.4.jar
[junit4:junit4]   2> 408 T75 oaz.Environment.logEnv Client environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
[junit4:junit4]   2> 408 T75 oaz.Environment.logEnv Client environment:java.io.tmpdir=.
[junit4:junit4]   2> 408 T75 oaz.Environment.logEnv Client environment:java.compiler=<NA>
[junit4:junit4]   2> 408 T75 oaz.Environment.logEnv Client environment:os.name=FreeBSD
[junit4:junit4]   2> 409 T75 oaz.Environment.logEnv Client environment:os.arch=amd64
[junit4:junit4]   2> 409 T75 oaz.Environment.logEnv Client environment:os.version=9.0-RELEASE
[junit4:junit4]   2> 409 T75 oaz.Environment.logEnv Client environment:user.name=hudson
[junit4:junit4]   2> 409 T75 oaz.Environment.logEnv Client environment:user.home=/home/hudson
[junit4:junit4]   2> 410 T75 oaz.Environment.logEnv Client environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 415 T75 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57157 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@1bcc1503
[junit4:junit4]   2> 469 T80 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:57157
[junit4:junit4]   2> 470 T75 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 472 T80 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57157, initiating session
[junit4:junit4]   2> 472 T77 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:21567
[junit4:junit4]   2> 485 T77 oazs.NIOServerCnxn.doIO WARNING Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
[junit4:junit4]   2> 485 T77 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:21567 (no session established for client)
[junit4:junit4]   2> 486 T80 oaz.ClientCnxn$SendThread.run Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
[junit4:junit4]   2> 2204 T80 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server localhost/127.0.0.1:57157
[junit4:junit4]   2> 2204 T80 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57157, initiating session
[junit4:junit4]   2> 2205 T77 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:21636
[junit4:junit4]   2> 2206 T77 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:21636
[junit4:junit4]   2> 2222 T79 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 2264 T79 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac3e1b9bc0000 with negotiated timeout 10000 for client /140.211.11.196:21636
[junit4:junit4]   2> 2265 T80 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:57157, sessionid = 0x13ac3e1b9bc0000, negotiated timeout = 10000
[junit4:junit4]   2> 2269 T82 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1bcc1503 name:ZooKeeperConnection Watcher:127.0.0.1:57157 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2270 T75 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2274 T75 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 2349 T81 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x13ac3e1b9bc0000
[junit4:junit4]   2> 2351 T82 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 2351 T75 oaz.ZooKeeper.close Session: 0x13ac3e1b9bc0000 closed
[junit4:junit4]   2> 2351 T75 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57157/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@7b0c2fa
[junit4:junit4]   2> 2352 T77 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:21636 which had sessionid 0x13ac3e1b9bc0000
[junit4:junit4]   2> 2352 T83 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:57157
[junit4:junit4]   2> 2352 T75 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2353 T83 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57157, initiating session
[junit4:junit4]   2> 2354 T77 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:43000
[junit4:junit4]   2> 2354 T77 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:43000
[junit4:junit4]   2> 2356 T79 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac3e1b9bc0001 with negotiated timeout 10000 for client /140.211.11.196:43000
[junit4:junit4]   2> 2356 T83 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:57157, sessionid = 0x13ac3e1b9bc0001, negotiated timeout = 10000
[junit4:junit4]   2> 2356 T84 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7b0c2fa name:ZooKeeperConnection Watcher:127.0.0.1:57157/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2357 T75 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2370 T75 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 2376 T75 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 2380 T75 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 2388 T75 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 2393 T75 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2423 T75 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2438 T75 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 2439 T75 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 2445 T75 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2445 T75 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2465 T75 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 2466 T75 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 2471 T75 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 2472 T75 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 2476 T75 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 2477 T75 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 2481 T75 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 2482 T75 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 2487 T75 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 2487 T75 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 2495 T75 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 2496 T75 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 2500 T75 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 2501 T75 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 2506 T81 oazs.PrepRequestProcessor.pRequest Processed session termination for sessionid: 0x13ac3e1b9bc0001
[junit4:junit4]   2> 2507 T84 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 2507 T75 oaz.ZooKeeper.close Session: 0x13ac3e1b9bc0001 closed
[junit4:junit4]   2> 2507 T77 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:43000 which had sessionid 0x13ac3e1b9bc0001
[junit4:junit4]   2> 2508 T75 oasc.LeaderElectionIntegrationTest.setUp ####SETUP_START testLeaderElectionAfterClientTimeout
[junit4:junit4]   2> 2510 T75 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2511 T75 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 2511 T75 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 2512 T75 oasc.CoreContainer.<init> New CoreContainer 1755608529
[junit4:junit4]   2> 2512 T75 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2513 T75 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 2590 T75 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 2607 T75 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:57157/solr
[junit4:junit4]   2> 2635 T75 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57157/solr sessionTimeout=3000 watcher=org.apache.solr.common.cloud.ConnectionManager@5e934dd
[junit4:junit4]   2> 2637 T85 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:57157
[junit4:junit4]   2> 2637 T75 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2638 T85 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57157, initiating session
[junit4:junit4]   2> 2638 T77 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:10260
[junit4:junit4]   2> 2639 T77 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:10260
[junit4:junit4]   2> 2640 T79 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac3e1b9bc0002 with negotiated timeout 3000 for client /140.211.11.196:10260
[junit4:junit4]   2> 2640 T85 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:57157, sessionid = 0x13ac3e1b9bc0002, negotiated timeout = 3000
[junit4:junit4]   2> 2641 T86 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5e934dd name:ZooKeeperConnection Watcher:127.0.0.1:57157/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2641 T75 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2663 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0002 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2675 T75 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 2678 T75 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:7000_solr
[junit4:junit4]   2> 2684 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0002 type:delete cxid:0x7 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:7000_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:7000_solr
[junit4:junit4]   2> 2685 T75 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:7000_solr
[junit4:junit4]   2> 2690 T75 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 2690 T75 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 2690 T75 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 2691 T75 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 2691 T75 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 2691 T75 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 2692 T75 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 2692 T75 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 2693 T75 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 2693 T75 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 2711 T75 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 2734 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0002 type:delete cxid:0x15 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 2735 T75 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 2739 T75 oasc.Overseer.start Overseer (id=88598516938309634-127.0.0.1:7000_solr-n_0000000000) starting
[junit4:junit4]   2> 2740 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0002 type:create cxid:0x1a zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2766 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0002 type:create cxid:0x1b zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2767 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0002 type:create cxid:0x1c zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2783 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0002 type:create cxid:0x1d zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2784 T88 oasc.OverseerCollectionProcessor.run Process current queue of collection messages
[junit4:junit4]   2> 2785 T75 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 2789 T75 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 2824 T87 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 2868 T75 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 2869 T75 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 2870 T75 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 2870 T75 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 2872 T75 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 2874 T75 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 2875 T75 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 2988 T75 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 3075 T75 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 3077 T75 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 3110 T75 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 3855 T75 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 3870 T75 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 3872 T75 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 3907 T75 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 3918 T75 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 3929 T75 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 3934 T75 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 3936 T75 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7000/
[junit4:junit4]   2> 3939 T75 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@54db16d7
[junit4:junit4]   2> 3939 T75 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7000/index/
[junit4:junit4]   2> 3940 T75 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7000/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 3940 T75 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7000/index forceNew:false
[junit4:junit4]   2> 3942 T75 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2eab8f37 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7c11d6fe),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 3942 T75 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 3943 T75 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 3945 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 3945 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 3946 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 3946 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 3947 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 3947 T75 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 3948 T75 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 3948 T75 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 3948 T75 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 3949 T75 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 3949 T75 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 3950 T75 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 3950 T75 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 3950 T75 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 3951 T75 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 3951 T75 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 3952 T75 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3952 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3952 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3953 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3953 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3954 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3954 T75 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 3954 T75 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 3955 T75 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 3963 T75 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 3967 T75 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 3968 T75 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 3970 T75 oass.SolrIndexSearcher.<init> Opening Searcher@75fd56dc main
[junit4:junit4]   2> 3970 T75 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 3971 T75 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 3971 T75 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 3971 T75 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 3977 T75 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 4029 T89 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@75fd56dc main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 4033 T75 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 4034 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0002 type:create cxid:0x56 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 4045 T75 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 4045 T75 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:7000/solr shardId:shard1
[junit4:junit4]   2> 4055 T75 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 4066 T75 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 4088 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0002 type:delete cxid:0x65 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 4089 T75 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 4090 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0002 type:create cxid:0x66 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4334 T87 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4335 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7000_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7000/solr"}
[junit4:junit4]   2> 4348 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0002 type:create cxid:0x6d zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 4384 T86 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (1)
[junit4:junit4]   2> 4602 T75 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 4603 T75 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 4603 T75 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:7000/solr/collection1/
[junit4:junit4]   2> 4603 T75 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 4609 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0002 type:create cxid:0x8b zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4892 T87 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4906 T86 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (1)
[junit4:junit4]   2> 4925 T75 oasc.ZkController.register We are http://127.0.0.1:7000/solr/collection1/ and leader is http://127.0.0.1:7000/solr/collection1/
[junit4:junit4]   2> 4925 T75 oasc.ZkController.checkRecovery WARNING Skipping recovery according to sys prop solrcloud.skip.autorecovery
[junit4:junit4]   2> 4926 T75 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 4929 T75 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4933 T75 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4933 T75 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 4933 T75 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 4934 T75 oasc.CoreContainer.<init> New CoreContainer 229421624
[junit4:junit4]   2> 4935 T75 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4935 T75 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 4999 T75 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 5016 T75 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:57157/solr
[junit4:junit4]   2> 5017 T75 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57157/solr sessionTimeout=3000 watcher=org.apache.solr.common.cloud.ConnectionManager@577670ee
[junit4:junit4]   2> 5018 T90 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:57157
[junit4:junit4]   2> 5018 T75 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 5019 T90 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57157, initiating session
[junit4:junit4]   2> 5019 T77 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:54083
[junit4:junit4]   2> 5020 T77 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:54083
[junit4:junit4]   2> 5021 T79 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac3e1b9bc0003 with negotiated timeout 3000 for client /140.211.11.196:54083
[junit4:junit4]   2> 5021 T90 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:57157, sessionid = 0x13ac3e1b9bc0003, negotiated timeout = 3000
[junit4:junit4]   2> 5022 T91 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@577670ee name:ZooKeeperConnection Watcher:127.0.0.1:57157/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 5022 T75 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 5023 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0003 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5024 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0003 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5026 T75 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:7001_solr
[junit4:junit4]   2> 5027 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0003 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:7001_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:7001_solr
[junit4:junit4]   2> 5034 T75 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:7001_solr
[junit4:junit4]   2> 5037 T86 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 5038 T75 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 5038 T75 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 5039 T75 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 5039 T75 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 5039 T75 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 5040 T75 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 5040 T75 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 5040 T75 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 5041 T75 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 5041 T75 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 5061 T75 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 5069 T75 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 5069 T75 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 5070 T75 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 5070 T75 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 5072 T75 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 5073 T75 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 5074 T75 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 5140 T75 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 5222 T75 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 5224 T75 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 5254 T75 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 5412 T87 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5413 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7000_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7000/solr"}
[junit4:junit4]   2> 5427 T86 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 5427 T91 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (2)
[junit4:junit4]   2> 5887 T75 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 5903 T75 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 5907 T75 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5933 T75 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5940 T75 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5949 T75 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5950 T75 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5952 T75 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7001/
[junit4:junit4]   2> 5953 T75 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@54db16d7
[junit4:junit4]   2> 5954 T75 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7001/index/
[junit4:junit4]   2> 5955 T75 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7001/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5956 T75 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7001/index forceNew:false
[junit4:junit4]   2> 5958 T75 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(NRTCachingDirectory(org.apache.lucene.store.RAMDirectory@66d9ce6e lockFactory=org.apache.lucene.store.NativeFSLockFactory@245b315a; maxCacheMB=0.923828125 maxMergeSizeMB=0.8994140625)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5959 T75 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 5960 T75 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 5962 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 5963 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 5963 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 5964 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 5965 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 5965 T75 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 5966 T75 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 5966 T75 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 5967 T75 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 5968 T75 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 5968 T75 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 5969 T75 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 5970 T75 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 5970 T75 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 5971 T75 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 5971 T75 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 5972 T75 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5973 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5973 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5974 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5975 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5975 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5976 T75 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5977 T75 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 5977 T75 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 5990 T75 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 5995 T75 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 5996 T75 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 5999 T75 oass.SolrIndexSearcher.<init> Opening Searcher@66a22ed7 main
[junit4:junit4]   2> 5999 T75 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 6000 T75 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 6001 T75 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 6001 T75 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 6024 T75 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 6084 T92 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@66a22ed7 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 6091 T75 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6093 T75 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 6093 T75 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:7001/solr shardId:shard1
[junit4:junit4]   2> 6098 T75 oasc.ZkController.register We are http://127.0.0.1:7001/solr/collection1/ and leader is http://127.0.0.1:7000/solr/collection1/
[junit4:junit4]   2> 6098 T75 oasc.ZkController.checkRecovery WARNING Skipping recovery according to sys prop solrcloud.skip.autorecovery
[junit4:junit4]   2> 6099 T75 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6101 T75 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6104 T75 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 6104 T75 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 6105 T75 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 6105 T75 oasc.CoreContainer.<init> New CoreContainer 649127729
[junit4:junit4]   2> 6106 T75 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 6106 T75 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 6167 T75 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 6182 T75 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:57157/solr
[junit4:junit4]   2> 6183 T75 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57157/solr sessionTimeout=3000 watcher=org.apache.solr.common.cloud.ConnectionManager@467746a2
[junit4:junit4]   2> 6184 T93 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:57157
[junit4:junit4]   2> 6184 T75 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 6185 T93 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57157, initiating session
[junit4:junit4]   2> 6186 T77 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:21278
[junit4:junit4]   2> 6186 T77 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:21278
[junit4:junit4]   2> 6188 T79 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac3e1b9bc0004 with negotiated timeout 3000 for client /140.211.11.196:21278
[junit4:junit4]   2> 6188 T93 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:57157, sessionid = 0x13ac3e1b9bc0004, negotiated timeout = 3000
[junit4:junit4]   2> 6188 T94 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@467746a2 name:ZooKeeperConnection Watcher:127.0.0.1:57157/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 6188 T75 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 6189 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0004 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6191 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0004 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6200 T75 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:7002_solr
[junit4:junit4]   2> 6200 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0004 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:7002_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:7002_solr
[junit4:junit4]   2> 6201 T75 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:7002_solr
[junit4:junit4]   2> 6205 T86 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 6205 T91 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 6206 T75 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 6207 T75 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 6207 T75 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 6207 T75 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 6208 T75 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 6208 T75 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 6208 T75 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 6209 T75 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 6209 T75 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 6210 T75 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 6232 T75 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 6247 T75 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 6248 T75 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 6249 T75 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 6249 T75 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 6251 T75 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 6253 T75 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 6253 T75 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 6314 T75 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 6393 T75 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 6395 T75 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 6415 T75 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 6448 T87 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6449 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7001_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7001/solr"}
[junit4:junit4]   2> 6482 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7001_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7001/solr"}
[junit4:junit4]   2> 6494 T86 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 6494 T91 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 6494 T94 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (3)
[junit4:junit4]   2> 7037 T75 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 7065 T75 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 7070 T75 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 7102 T75 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 7108 T75 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 7114 T75 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 7116 T75 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 7118 T75 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7002/
[junit4:junit4]   2> 7118 T75 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@54db16d7
[junit4:junit4]   2> 7120 T75 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7002/index/
[junit4:junit4]   2> 7120 T75 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7002/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 7121 T75 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7002/index forceNew:false
[junit4:junit4]   2> 7123 T75 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@14dc1f90 lockFactory=org.apache.lucene.store.NativeFSLockFactory@677ec421),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 7124 T75 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 7124 T75 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 7128 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 7128 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 7129 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 7129 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 7130 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 7131 T75 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 7131 T75 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 7132 T75 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 7133 T75 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 7134 T75 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 7134 T75 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 7135 T75 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 7136 T75 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 7136 T75 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 7137 T75 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 7138 T75 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 7138 T75 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7139 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7140 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7140 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7141 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7142 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7143 T75 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7143 T75 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 7144 T75 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 7157 T75 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 7162 T75 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 7163 T75 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 7166 T75 oass.SolrIndexSearcher.<init> Opening Searcher@2cb056c8 main
[junit4:junit4]   2> 7166 T75 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.BaseDirectoryWrapper
[junit4:junit4]   2> 7167 T75 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 7167 T75 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 7168 T75 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 7202 T75 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 7262 T95 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2cb056c8 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 7269 T75 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 7272 T75 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 7272 T75 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:7002/solr shardId:shard1
[junit4:junit4]   2> 7276 T75 oasc.ZkController.register We are http://127.0.0.1:7002/solr/collection1/ and leader is http://127.0.0.1:7000/solr/collection1/
[junit4:junit4]   2> 7277 T75 oasc.ZkController.checkRecovery WARNING Skipping recovery according to sys prop solrcloud.skip.autorecovery
[junit4:junit4]   2> 7277 T75 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 7280 T75 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 7283 T75 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 7283 T75 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 7284 T75 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 7284 T75 oasc.CoreContainer.<init> New CoreContainer 1379550442
[junit4:junit4]   2> 7285 T75 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 7285 T75 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 7347 T75 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 7363 T75 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:57157/solr
[junit4:junit4]   2> 7363 T75 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57157/solr sessionTimeout=3000 watcher=org.apache.solr.common.cloud.ConnectionManager@6be7b7cd
[junit4:junit4]   2> 7364 T96 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:57157
[junit4:junit4]   2> 7364 T75 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7366 T96 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57157, initiating session
[junit4:junit4]   2> 7366 T77 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:35865
[junit4:junit4]   2> 7366 T77 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:35865
[junit4:junit4]   2> 7368 T79 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac3e1b9bc0005 with negotiated timeout 3000 for client /140.211.11.196:35865
[junit4:junit4]   2> 7368 T96 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:57157, sessionid = 0x13ac3e1b9bc0005, negotiated timeout = 3000
[junit4:junit4]   2> 7369 T97 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6be7b7cd name:ZooKeeperConnection Watcher:127.0.0.1:57157/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7369 T75 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7370 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0005 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7371 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0005 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7373 T75 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:7003_solr
[junit4:junit4]   2> 7373 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0005 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:7003_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:7003_solr
[junit4:junit4]   2> 7382 T75 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:7003_solr
[junit4:junit4]   2> 7385 T86 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 7385 T91 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 7385 T94 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 7386 T75 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 7386 T75 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 7387 T75 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 7387 T75 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 7387 T75 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 7388 T75 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 7388 T75 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 7388 T75 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 7389 T75 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 7389 T75 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 7411 T75 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 7418 T75 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 7418 T75 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 7419 T75 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 7420 T75 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 7421 T75 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 7423 T75 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 7424 T75 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 7480 T75 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 7513 T87 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 7514 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7002_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7002/solr"}
[junit4:junit4]   2> 7521 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7002_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7002/solr"}
[junit4:junit4]   2> 7534 T86 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 7534 T94 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 7534 T91 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 7534 T97 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 7552 T75 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 7554 T75 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 7570 T75 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 8182 T75 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 8197 T75 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 8201 T75 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 8226 T75 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8232 T75 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8237 T75 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8239 T75 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8241 T75 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7003/
[junit4:junit4]   2> 8241 T75 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@54db16d7
[junit4:junit4]   2> 8242 T75 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7003/index/
[junit4:junit4]   2> 8243 T75 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7003/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 8244 T75 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7003/index forceNew:false
[junit4:junit4]   2> 8246 T75 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@49fc1d7e lockFactory=org.apache.lucene.store.NativeFSLockFactory@5615a6e0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 8247 T75 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 8247 T75 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 8250 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 8251 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 8251 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 8251 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 8252 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 8253 T75 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 8253 T75 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 8254 T75 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 8255 T75 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 8255 T75 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 8256 T75 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 8257 T75 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 8257 T75 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 8258 T75 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 8258 T75 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 8259 T75 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 8259 T75 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8260 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8261 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8261 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8262 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8263 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8263 T75 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8264 T75 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 8265 T75 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 8277 T75 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 8281 T75 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 8283 T75 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 8286 T75 oass.SolrIndexSearcher.<init> Opening Searcher@68dd9142 main
[junit4:junit4]   2> 8286 T75 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 8287 T75 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 8287 T75 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 8288 T75 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 8337 T75 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 8492 T98 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@68dd9142 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 8498 T75 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8523 T75 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 8523 T75 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:7003/solr shardId:shard1
[junit4:junit4]   2> 8532 T75 oasc.ZkController.register We are http://127.0.0.1:7003/solr/collection1/ and leader is http://127.0.0.1:7000/solr/collection1/
[junit4:junit4]   2> 8532 T75 oasc.ZkController.checkRecovery WARNING Skipping recovery according to sys prop solrcloud.skip.autorecovery
[junit4:junit4]   2> 8533 T75 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8555 T75 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8557 T87 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8558 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7003_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7003/solr"}
[junit4:junit4]   2> 8559 T75 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 8559 T75 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 8560 T75 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 8560 T75 oasc.CoreContainer.<init> New CoreContainer 133269757
[junit4:junit4]   2> 8561 T75 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 8561 T75 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 8624 T75 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 8627 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7003_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7003/solr"}
[junit4:junit4]   2> 8640 T75 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:57157/solr
[junit4:junit4]   2> 8641 T75 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57157/solr sessionTimeout=3000 watcher=org.apache.solr.common.cloud.ConnectionManager@6ceb8bfa
[junit4:junit4]   2> 8642 T99 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:57157
[junit4:junit4]   2> 8642 T75 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 8643 T99 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57157, initiating session
[junit4:junit4]   2> 8643 T77 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:13862
[junit4:junit4]   2> 8644 T77 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:13862
[junit4:junit4]   2> 8664 T86 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 8664 T91 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 8664 T97 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 8664 T94 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (4)
[junit4:junit4]   2> 8665 T79 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac3e1b9bc0006 with negotiated timeout 3000 for client /140.211.11.196:13862
[junit4:junit4]   2> 8665 T99 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:57157, sessionid = 0x13ac3e1b9bc0006, negotiated timeout = 3000
[junit4:junit4]   2> 8665 T100 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6ceb8bfa name:ZooKeeperConnection Watcher:127.0.0.1:57157/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 8666 T75 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 8667 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0006 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8668 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0006 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8669 T75 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:7004_solr
[junit4:junit4]   2> 8670 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0006 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:7004_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:7004_solr
[junit4:junit4]   2> 8671 T75 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:7004_solr
[junit4:junit4]   2> 8674 T86 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 8674 T97 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 8674 T91 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 8674 T94 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 8675 T75 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 8676 T75 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 8676 T75 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 8676 T75 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 8677 T75 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 8677 T75 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 8677 T75 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 8678 T75 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 8678 T75 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 8678 T75 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 8699 T75 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 8706 T75 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 8706 T75 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 8707 T75 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 8708 T75 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 8709 T75 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 8711 T75 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 8711 T75 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 8764 T75 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 8840 T75 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 8842 T75 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 8856 T75 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 9409 T75 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 9422 T75 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 9425 T75 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 9445 T75 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9450 T75 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9454 T75 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9455 T75 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9457 T75 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7004/
[junit4:junit4]   2> 9457 T75 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@54db16d7
[junit4:junit4]   2> 9458 T75 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7004/index/
[junit4:junit4]   2> 9458 T75 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7004/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 9459 T75 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_7004/index forceNew:false
[junit4:junit4]   2> 9461 T75 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@119cb48 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4fabbfd2),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 9461 T75 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 9461 T75 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 9463 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 9464 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 9464 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 9464 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 9465 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 9465 T75 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 9466 T75 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 9466 T75 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 9467 T75 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 9467 T75 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 9468 T75 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 9468 T75 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 9469 T75 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 9469 T75 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 9470 T75 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 9470 T75 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 9471 T75 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9471 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9472 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9472 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9473 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9473 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9474 T75 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9474 T75 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 9475 T75 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 9483 T75 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 9486 T75 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 9488 T75 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 9490 T75 oass.SolrIndexSearcher.<init> Opening Searcher@4242dfb main
[junit4:junit4]   2> 9490 T75 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 9490 T75 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 9491 T75 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 9491 T75 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 9523 T75 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 9576 T101 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4242dfb main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 9580 T75 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9582 T75 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 9582 T75 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:7004/solr shardId:shard1
[junit4:junit4]   2> 9586 T75 oasc.ZkController.register We are http://127.0.0.1:7004/solr/collection1/ and leader is http://127.0.0.1:7000/solr/collection1/
[junit4:junit4]   2> 9586 T75 oasc.ZkController.checkRecovery WARNING Skipping recovery according to sys prop solrcloud.skip.autorecovery
[junit4:junit4]   2> 9586 T75 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9593 T75 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9596 T75 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 9596 T75 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr
[junit4:junit4]   2> 9596 T75 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4]   2> 9596 T75 oasc.CoreContainer.<init> New CoreContainer 108824254
[junit4:junit4]   2> 9597 T75 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 9597 T75 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4]   2> 9636 T75 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 9646 T75 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:57157/solr
[junit4:junit4]   2> 9647 T75 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57157/solr sessionTimeout=3000 watcher=org.apache.solr.common.cloud.ConnectionManager@3111c576
[junit4:junit4]   2> 9648 T102 oaz.ClientCnxn$SendThread.startConnect Opening socket connection to server /127.0.0.1:57157
[junit4:junit4]   2> 9648 T75 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 9649 T102 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57157, initiating session
[junit4:junit4]   2> 9649 T77 oazs.NIOServerCnxn$Factory.run Accepted socket connection from /140.211.11.196:21843
[junit4:junit4]   2> 9649 T77 oazs.NIOServerCnxn.readConnectRequest Client attempting to establish new session at /140.211.11.196:21843
[junit4:junit4]   2> 9651 T79 oazs.NIOServerCnxn.finishSessionInit Established session 0x13ac3e1b9bc0007 with negotiated timeout 3000 for client /140.211.11.196:21843
[junit4:junit4]   2> 9651 T102 oaz.ClientCnxn$SendThread.readConnectResult Session establishment complete on server localhost/127.0.0.1:57157, sessionid = 0x13ac3e1b9bc0007, negotiated timeout = 3000
[junit4:junit4]   2> 9652 T103 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3111c576 name:ZooKeeperConnection Watcher:127.0.0.1:57157/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 9652 T75 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 9653 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0007 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9654 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0007 type:create cxid:0x2 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9655 T75 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:3333_solr
[junit4:junit4]   2> 9656 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0007 type:delete cxid:0x4 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:3333_solr Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:3333_solr
[junit4:junit4]   2> 9665 T75 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:3333_solr
[junit4:junit4]   2> 9668 T86 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 9669 T91 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 9669 T97 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 9668 T94 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 9669 T100 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 9669 T75 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4]   2> 9670 T75 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 9670 T75 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4]   2> 9670 T75 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 9670 T75 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 9671 T75 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 9671 T75 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 9671 T75 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 9671 T75 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 9672 T75 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4]   2> 9677 T87 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9677 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7004_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7004/solr"}
[junit4:junit4]   2> 9682 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:7004_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:7004/solr"}
[junit4:junit4]   2> 9692 T75 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 9693 T86 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 9693 T97 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 9693 T91 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 9693 T100 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 9693 T94 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 9693 T103 oaz.ClientCnxn$EventThread.processEvent SEVERE Error while calling watcher  java.lang.NullPointerException
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader$2.process(ZkStateReader.java:189)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:526)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
[junit4:junit4]   2> 
[junit4:junit4]   2> 9699 T75 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4]   2> 9699 T75 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 9700 T75 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 9700 T75 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 9701 T75 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4]   2> 9702 T75 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 9703 T75 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4]   2> 9740 T75 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_41
[junit4:junit4]   2> 9788 T75 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 9789 T75 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 9799 T75 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 10235 T75 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 10244 T75 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 10247 T75 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 10449 T75 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10454 T75 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10458 T75 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10459 T75 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10461 T75 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_3333/
[junit4:junit4]   2> 10461 T75 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@54db16d7
[junit4:junit4]   2> 10462 T75 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_3333/index/
[junit4:junit4]   2> 10462 T75 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_3333/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 10463 T75 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build/solr-core/test/J0/./solrtest-LeaderElectionIntegrationTest-1351906080841/data_3333/index forceNew:false
[junit4:junit4]   2> 10465 T75 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@19d0954e lockFactory=org.apache.lucene.store.NativeFSLockFactory@41c981b4),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 10465 T75 oasc.SolrDeletionPolicy.updateCommits newest commit = 1
[junit4:junit4]   2> 10466 T75 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 10468 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 10469 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 10469 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 10469 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 10470 T75 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 10470 T75 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 10471 T75 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 10471 T75 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 10472 T75 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 10472 T75 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 10473 T75 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 10473 T75 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 10474 T75 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 10474 T75 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 10474 T75 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 10475 T75 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 10475 T75 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10476 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10476 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10477 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10477 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10478 T75 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10478 T75 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10479 T75 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 10479 T75 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 10488 T75 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 10491 T75 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 10492 T75 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 10495 T75 oass.SolrIndexSearcher.<init> Opening Searcher@36800134 main
[junit4:junit4]   2> 10495 T75 oass.SolrIndexSearcher.getIndexDir WARNING WARNING: Directory impl does not support setting indexDir: org.apache.lucene.store.MockDirectoryWrapper
[junit4:junit4]   2> 10495 T75 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 10496 T75 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 10496 T75 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 10541 T75 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 10586 T104 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@36800134 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 10593 T75 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 10595 T75 oasc.CoreContainer.register registering core: collection1
[junit4:junit4]   2> 10595 T75 oasc.ZkController.register Register shard - core:collection1 address:http://127.0.0.1:3333/solr shardId:shard2
[junit4:junit4]   2> 10596 T75 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 10610 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0007 type:delete cxid:0x40 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 10611 T75 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 10612 T81 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ac3e1b9bc0007 type:create cxid:0x41 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10704 T87 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 10704 T87 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":null,
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:3333_solr",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:3333/solr"}
[junit4:junit4]   2> 10722 T91 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 10722 T86 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 10723 T94 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 10723 T100 oascc.ZkStateReader$2.process A cluster state change has occurred - updating... (6)
[junit4:junit4]   2> 107

[...truncated too long message...]

ve_errors=0}
[junit4:junit4]   2> 7222870 T75 oasc.SolrCore.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4]   2> 7222870 T75 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4]   2> 7222871 T75 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4]   2> 7222871 T75 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4]   2> 7223297 T102 oaz.ClientCnxn$SendThread.run Client session timed out, have not heard from server in 3000ms for sessionid 0x13ac3e1b9bc0007, closing socket connection and attempting reconnect
[junit4:junit4]   2> 7223342 T75 oaz.ZooKeeper.close Session: 0x13ac3e1b9bc0006 closed
[junit4:junit4]   2> 7223342 T75 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=108824254
[junit4:junit4]   2> 7223343 T75 oasc.SolrCore.close SEVERE Too many close [count:-1] on org.apache.solr.core.SolrCore@5bb8ad39. Please report this exception to solr-user@lucene.apache.org
[junit4:junit4]   2> 7223371 T74 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
[junit4:junit4]   2> 	   1) Thread[id=102, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[74596681D419C368]-SendThread(localhost:57157), state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	        at java.lang.Thread.sleep(Native Method)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1038)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1101)
[junit4:junit4]   2> 	   2) Thread[id=75, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[74596681D419C368], state=WAITING, group=TGRP-LeaderElectionIntegrationTest]
[junit4:junit4]   2> 	        at java.lang.Object.wait(Native Method)
[junit4:junit4]   2> 	        at java.lang.Object.wait(Object.java:503)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1345)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn.close(ClientCnxn.java:1323)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ZooKeeper.close(ZooKeeper.java:537)
[junit4:junit4]   2> 	        at org.apache.solr.common.cloud.SolrZooKeeper.close(SolrZooKeeper.java:91)
[junit4:junit4]   2> 	        at org.apache.solr.common.cloud.SolrZkClient.closeKeeper(SolrZkClient.java:573)
[junit4:junit4]   2> 	        at org.apache.solr.common.cloud.SolrZkClient.close(SolrZkClient.java:542)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.ZkController.close(ZkController.java:319)
[junit4:junit4]   2> 	        at org.apache.solr.core.CoreContainer.shutdown(CoreContainer.java:603)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.LeaderElectionIntegrationTest.tearDown(LeaderElectionIntegrationTest.java:289)
[junit4:junit4]   2> 	        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4]   2> 	        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4:junit4]   2> 	        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4:junit4]   2> 	        at java.lang.reflect.Method.invoke(Method.java:601)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:795)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene40, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=el_GR, timezone=America/Cancun
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=6,free=90418984,total=145752064
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestFaceting, TestGroupingSearch, DistanceFunctionTest, LeaderElectionIntegrationTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderElectionIntegrationTest -Dtests.seed=74596681D419C368 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=el_GR -Dtests.timezone=America/Cancun -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] ERROR   0.00s J0 | LeaderElectionIntegrationTest (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([74596681D419C368]:0)
[junit4:junit4] Completed on J0 in 7223.42s, 1 test, 2 errors <<< FAILURES!

[...truncated 174 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:335: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/module-build.xml:61: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:1163: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:827: There were test failures: 233 suites, 964 tests, 3 suite-level errors, 1 error, 850 ignored

Total time: 137 minutes 58 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure