You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2015/06/10 09:22:58 UTC

[JENKINS] Lucene-Solr-5.x-Linux (64bit/jdk1.9.0-ea-b60) - Build # 12833 - Failure!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Linux/12833/
Java: 64bit/jdk1.9.0-ea-b60 -XX:+UseCompressedOops -XX:+UseParallelGC

3 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.TestDistributedGrouping

Error Message:
2 threads leaked from SUITE scope at org.apache.solr.TestDistributedGrouping:     1) Thread[id=7553, name=searcherExecutor-3689-thread-1, state=WAITING, group=TGRP-TestDistributedGrouping]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=7545, name=qtp70567688-7545, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at java.util.WeakHashMap.get(WeakHashMap.java:403)         at org.apache.solr.servlet.cache.HttpCacheHeaderUtil.calcEtag(HttpCacheHeaderUtil.java:101)         at org.apache.solr.servlet.cache.HttpCacheHeaderUtil.doCacheHeaderValidation(HttpCacheHeaderUtil.java:219)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:444)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:106)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:364)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:497)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.TestDistributedGrouping: 
   1) Thread[id=7553, name=searcherExecutor-3689-thread-1, state=WAITING, group=TGRP-TestDistributedGrouping]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=7545, name=qtp70567688-7545, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at java.util.WeakHashMap.get(WeakHashMap.java:403)
        at org.apache.solr.servlet.cache.HttpCacheHeaderUtil.calcEtag(HttpCacheHeaderUtil.java:101)
        at org.apache.solr.servlet.cache.HttpCacheHeaderUtil.doCacheHeaderValidation(HttpCacheHeaderUtil.java:219)
        at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:444)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
        at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:106)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
        at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)
        at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:364)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
        at org.eclipse.jetty.server.Server.handle(Server.java:497)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
        at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([9C7CF3E7DBC4FE77]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.TestDistributedGrouping

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=7553, name=searcherExecutor-3689-thread-1, state=WAITING, group=TGRP-TestDistributedGrouping]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=7545, name=qtp70567688-7545, state=RUNNABLE, group=TGRP-TestDistributedGrouping]         at java.util.WeakHashMap.get(WeakHashMap.java:403)         at org.apache.solr.servlet.cache.HttpCacheHeaderUtil.calcEtag(HttpCacheHeaderUtil.java:101)         at org.apache.solr.servlet.cache.HttpCacheHeaderUtil.doCacheHeaderValidation(HttpCacheHeaderUtil.java:219)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:444)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:106)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:364)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)         at org.eclipse.jetty.server.Server.handle(Server.java:497)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=7553, name=searcherExecutor-3689-thread-1, state=WAITING, group=TGRP-TestDistributedGrouping]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=7545, name=qtp70567688-7545, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
        at java.util.WeakHashMap.get(WeakHashMap.java:403)
        at org.apache.solr.servlet.cache.HttpCacheHeaderUtil.calcEtag(HttpCacheHeaderUtil.java:101)
        at org.apache.solr.servlet.cache.HttpCacheHeaderUtil.doCacheHeaderValidation(HttpCacheHeaderUtil.java:219)
        at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:444)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
        at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:106)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
        at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)
        at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:364)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
        at org.eclipse.jetty.server.Server.handle(Server.java:497)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
        at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([9C7CF3E7DBC4FE77]:0)


FAILED:  org.apache.solr.TestDistributedGrouping.test

Error Message:
Timeout occured while waiting response from server at: https://127.0.0.1:48813/oba/collection1

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:48813/oba/collection1
	at __randomizedtesting.SeedInfo.seed([9C7CF3E7DBC4FE77:1428CC3D7538938F]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:572)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
	at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:600)
	at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:588)
	at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:567)
	at org.apache.solr.TestDistributedGrouping.test(TestDistributedGrouping.java:59)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:502)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsRepeatStatement.callStatement(BaseDistributedSearchTestCase.java:982)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:170)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
	at sun.security.ssl.InputRecord.read(InputRecord.java:503)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:961)
	at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:918)
	at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:466)
	... 50 more




Build Log:
[...truncated 10905 lines...]
   [junit4] Suite: org.apache.solr.TestDistributedGrouping
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/init-core-data-001
   [junit4]   2> 903889 INFO  (SUITE-TestDistributedGrouping-seed#[9C7CF3E7DBC4FE77]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 903889 INFO  (SUITE-TestDistributedGrouping-seed#[9C7CF3E7DBC4FE77]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /oba/
   [junit4]   2> 903933 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/control/cores/collection1
   [junit4]   2> 903935 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.e.j.s.Server jetty-9.2.10.v20150310
   [junit4]   2> 903935 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@531948dc{/oba,null,AVAILABLE}
   [junit4]   2> 903948 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2ea2c711{SSL-http/1.1}{127.0.0.1:54635}
   [junit4]   2> 903949 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.e.j.s.Server Started @906007ms
   [junit4]   2> 903949 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/oba, hostPort=54635, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/control/cores}
   [junit4]   2> 903950 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@3b764bce
   [junit4]   2> 903950 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/control/'
   [junit4]   2> 903958 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/control/solr.xml
   [junit4]   2> 903961 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/control/cores
   [junit4]   2> 903961 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.c.CoreContainer New CoreContainer 621763283
   [junit4]   2> 903961 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/control/]
   [junit4]   2> 903961 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.c.CoreContainer loading shared library: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/control/lib
   [junit4]   2> 903961 WARN  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/control/lib).
   [junit4]   2> 903965 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : https,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 903967 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 903967 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 903968 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 903968 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 903968 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.c.CoresLocator Looking for core definitions underneath /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/control/cores
   [junit4]   2> 903969 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/control/cores/collection1, absoluteInstDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/control/cores/collection1/, configSet=collection1, dataDir=data/}
   [junit4]   2> 903969 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.c.CoresLocator Found core collection1 in /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/control/cores/collection1/
   [junit4]   2> 903969 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 903970 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 903970 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 903970 INFO  (coreLoadExecutor-3633-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 903970 INFO  (coreLoadExecutor-3633-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 903970 INFO  (coreLoadExecutor-3633-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 903985 INFO  (coreLoadExecutor-3633-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 903994 INFO  (coreLoadExecutor-3633-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.3.0
   [junit4]   2> 904007 INFO  (coreLoadExecutor-3633-thread-1) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 904007 INFO  (coreLoadExecutor-3633-thread-1) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml
   [junit4]   2> 904009 INFO  (coreLoadExecutor-3633-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 904061 INFO  (coreLoadExecutor-3633-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 904064 INFO  (coreLoadExecutor-3633-thread-1) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 904064 INFO  (coreLoadExecutor-3633-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 904067 INFO  (coreLoadExecutor-3633-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 904068 INFO  (coreLoadExecutor-3633-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 904069 INFO  (coreLoadExecutor-3633-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 904069 WARN  (coreLoadExecutor-3633-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 904069 WARN  (coreLoadExecutor-3633-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 904069 INFO  (coreLoadExecutor-3633-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 904069 WARN  (coreLoadExecutor-3633-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 904069 WARN  (coreLoadExecutor-3633-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 904069 INFO  (coreLoadExecutor-3633-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from configset /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1
   [junit4]   2> 904070 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 904070 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/], dataDir=[null]
   [junit4]   2> 904070 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cdfb29f
   [junit4]   2> 904070 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/control/cores/collection1/data
   [junit4]   2> 904070 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/control/cores/collection1/data/index/
   [junit4]   2> 904070 WARN  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/control/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 904070 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/control/cores/collection1/data/index
   [junit4]   2> 904070 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=36, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=87.8583984375, floorSegmentMB=1.091796875, forceMergeDeletesPctAllowed=10.261147520847556, segmentsPerTier=16.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 904071 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6c628830 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@74b4ca0),segFN=segments_1,generation=1}
   [junit4]   2> 904071 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 904071 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/control/cores/collection1/data/
   [junit4]   2> 904071 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.DirectoryFactory /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/control/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 904073 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 904073 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 904073 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 904073 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 904073 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 904073 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 904074 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 904074 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 904074 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 904074 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 904074 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 904074 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 904075 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 904076 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 904076 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 904076 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 904078 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 904079 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,spellCheckCompRH,lazy,/update/json/docs,/admin/luke,spellCheckWithWordbreak_Direct,tvrh,/get,/admin/properties,/update/json,mltrh,/admin/threads,/search-facet-def,/update/csv,/search-facet-invariants,/admin/segments,spellCheckCompRH1,/mlt,/admin/system,/replication,dismax,defaults,/config,/schema,/admin/plugins,/admin/logging,spellCheckWithWordbreak,mock,/admin/ping,/update,/admin/file,/terms,spellCheckCompRH_Direct,/debug/dump
   [junit4]   2> 904079 WARN  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.c.SolrCore solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 904080 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 904080 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 904080 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 904081 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 904081 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 904081 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=13, maxMergeAtOnceExplicit=17, maxMergedSegmentMB=77.130859375, floorSegmentMB=0.658203125, forceMergeDeletesPctAllowed=19.024449019907134, segmentsPerTier=30.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 904081 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@6c628830 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@74b4ca0),segFN=segments_1,generation=1}
   [junit4]   2> 904082 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 904082 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@6867c4c[collection1] main
   [junit4]   2> 904082 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
   [junit4]   2> 904082 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 904082 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 904082 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 904094 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 904167 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 904167 INFO  (searcherExecutor-3634-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 904168 INFO  (searcherExecutor-3634-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 904168 INFO  (searcherExecutor-3634-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 904168 INFO  (searcherExecutor-3634-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 904168 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 904168 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 904168 INFO  (searcherExecutor-3634-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 904168 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 904168 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1503574909142433792
   [junit4]   2> 904168 INFO  (searcherExecutor-3634-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 904169 INFO  (searcherExecutor-3634-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 904169 INFO  (searcherExecutor-3634-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 904169 INFO  (searcherExecutor-3634-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 904169 INFO  (searcherExecutor-3634-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6867c4c[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 904170 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.u.UpdateLog Took 2 ms to seed version buckets with highest version 1503574909142433792
   [junit4]   2> 904170 INFO  (coreLoadExecutor-3633-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 904513 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/shard0/cores/collection1
   [junit4]   2> 904514 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.e.j.s.Server jetty-9.2.10.v20150310
   [junit4]   2> 904515 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@111b06db{/oba,null,AVAILABLE}
   [junit4]   2> 904520 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.e.j.s.ServerConnector Started ServerConnector@7f35cb66{SSL-http/1.1}{127.0.0.1:46822}
   [junit4]   2> 904520 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.e.j.s.Server Started @906578ms
   [junit4]   2> 904520 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/oba, hostPort=46822, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/shard0/cores}
   [junit4]   2> 904521 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@3b764bce
   [junit4]   2> 904521 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/shard0/'
   [junit4]   2> 904533 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/shard0/solr.xml
   [junit4]   2> 904537 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.c.CoresLocator Config-defined core root directory: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/shard0/cores
   [junit4]   2> 904537 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.c.CoreContainer New CoreContainer 160304595
   [junit4]   2> 904538 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/shard0/]
   [junit4]   2> 904538 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.c.CoreContainer loading shared library: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/shard0/lib
   [junit4]   2> 904538 WARN  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to classloader: lib (resolved as: /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/shard0/lib).
   [junit4]   2> 904543 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : https,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 904545 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 904546 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 904546 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 904547 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 904547 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.c.CoresLocator Looking for core definitions underneath /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/shard0/cores
   [junit4]   2> 904556 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.c.SolrCore Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/shard0/cores/collection1, absoluteInstDir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/shard0/cores/collection1/, configSet=collection1, dataDir=data/}
   [junit4]   2> 904556 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.c.CoresLocator Found core collection1 in /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/shard0/cores/collection1/
   [junit4]   2> 904556 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 904556 INFO  (coreLoadExecutor-3644-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 904556 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 904557 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 904557 INFO  (coreLoadExecutor-3644-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 904557 INFO  (coreLoadExecutor-3644-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 904566 INFO  (coreLoadExecutor-3644-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 904571 INFO  (coreLoadExecutor-3644-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.3.0
   [junit4]   2> 904583 INFO  (coreLoadExecutor-3644-thread-1) [    ] o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 904583 INFO  (coreLoadExecutor-3644-thread-1) [    ] o.a.s.s.IndexSchema Reading Solr Schema from /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml
   [junit4]   2> 904586 INFO  (coreLoadExecutor-3644-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 904638 INFO  (coreLoadExecutor-3644-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 904641 INFO  (coreLoadExecutor-3644-thread-1) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 904642 INFO  (coreLoadExecutor-3644-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 904644 INFO  (coreLoadExecutor-3644-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 904645 INFO  (coreLoadExecutor-3644-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 904646 INFO  (coreLoadExecutor-3644-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 904646 WARN  (coreLoadExecutor-3644-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 904646 WARN  (coreLoadExecutor-3644-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 904646 INFO  (coreLoadExecutor-3644-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 904646 WARN  (coreLoadExecutor-3644-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 904646 WARN  (coreLoadExecutor-3644-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 904646 INFO  (coreLoadExecutor-3644-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from configset /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1
   [junit4]   2> 904647 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 904647 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/], dataDir=[null]
   [junit4]   2> 904647 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cdfb29f
   [junit4]   2> 904647 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/shard0/cores/collection1/data
   [junit4]   2> 904647 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/shard0/cores/collection1/data/index/
   [junit4]   2> 904647 WARN  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/shard0/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 904647 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/shard0/cores/collection1/data/index
   [junit4]   2> 904648 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=36, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=87.8583984375, floorSegmentMB=1.091796875, forceMergeDeletesPctAllowed=10.261147520847556, segmentsPerTier=16.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 904648 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@cff7d5f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@d07765b),segFN=segments_1,generation=1}
   [junit4]   2> 904648 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 904648 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/shard0/cores/collection1/data/
   [junit4]   2> 904648 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.DirectoryFactory /home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.TestDistributedGrouping_9C7CF3E7DBC4FE77-001/tempDir-001/shard0/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 904650 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 904650 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 904650 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 904650 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 904651 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 904651 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 904651 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 904651 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 904651 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 904651 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 904651 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 904651 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 904652 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 904653 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 904653 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 904653 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 904656 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 904657 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,spellCheckCompRH,lazy,/update/json/docs,/admin/luke,spellCheckWithWordbreak_Direct,tvrh,/get,/admin/properties,/update/json,mltrh,/admin/threads,/search-facet-def,/update/csv,/search-facet-invariants,/admin/segments,spellCheckCompRH1,/mlt,/admin/system,/replication,dismax,defaults,/config,/schema,/admin/plugins,/admin/logging,spellCheckWithWordbreak,mock,/admin/ping,/update,/admin/file,/terms,spellCheckCompRH_Direct,/debug/dump
   [junit4]   2> 904657 WARN  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.c.SolrCore solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 904657 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 904658 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 904658 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 904661 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 904661 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 904661 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=13, maxMergeAtOnceExplicit=17, maxMergedSegmentMB=77.130859375, floorSegmentMB=0.658203125, forceMergeDeletesPctAllowed=19.024449019907134, segmentsPerTier=30.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 904661 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@cff7d5f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@d07765b),segFN=segments_1,generation=1}
   [junit4]   2> 904661 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 904661 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@267f18cf[collection1] main
   [junit4]   2> 904662 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
   [junit4]   2> 904662 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 904662 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 904662 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 904674 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 904733 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 904733 INFO  (searcherExecutor-3645-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 904734 INFO  (searcherExecutor-3645-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 904734 INFO  (searcherExecutor-3645-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 904734 INFO  (searcherExecutor-3645-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 904734 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 904734 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 904734 INFO  (searcherExecutor-3645-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 904734 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 904734 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1503574909735927808
   [junit4]   2> 904734 INFO  (searcherExecutor-3645-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 904735 INFO  (searcherExecutor-3645-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 904735 INFO  (searcherExecutor-3645-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 904735 INFO  (searcherExecutor-3645-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 904736 INFO  (searcherExecutor-3645-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@267f18cf[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 904736 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.u.UpdateLog Took 2 ms to seed version buckets with highest version 1503574909735927808
   [junit4]   2> 904736 INFO  (coreLoadExecutor-3644-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 905058 INFO  (TEST-TestDistributedGrouping.test-seed#[9C7CF3E7DBC4FE77]) [    ] o.a.s.SolrTestCaseJ4 ###Starting test
   [junit4]   2> 905128 INFO  (qtp2053162384-7433) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1503574910148018176)} 0 1
   [junit4]   2> 905195 INFO  (qtp1400273651-7451) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1503574910218272768)} 0 1
   [junit4]   2> 905198 INFO  (qtp2053162384-7434) [    x:collection1] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 905198 INFO  (qtp2053162384-7434) [    x:collection1] o.a.s.u.UpdateHandler No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 905198 INFO  (qtp2053162384-7434) [    x:collection1] o.a.s.c.SolrCore SolrIndexSearcher has not changed - not re-opening: org.apache.solr.search.SolrIndexSearcher
   [junit4]   2> 905198 INFO  (qtp2053162384-7434) [    x:collection1] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 905198 INFO  (qtp2053162384-7434) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} {commit=} 0 1
   [junit4]   2> 905200 INFO  (qtp1400273651-7452) [    x:collection1] o.a.s.u.UpdateHandler start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 905201 INFO  (qtp1400273651-7452) [    x:collection1] o.a.s.u.UpdateHandler No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 905201 INFO  (qtp1400273651-7452) [    x:collection1] o.a.s.c.SolrCore SolrIndexSearcher has not changed - not re-opening: org.apache.solr.search.SolrIndexSearcher
   [junit4]   2> 905201 INFO  (qtp1400273651-7452) [    x:collection1] o.a.s.u.UpdateHandler end_commit_flush
   [junit4]   2> 905201 INFO  (qtp1400273651-7452) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} {commit=} 0 1
   [junit4]   2> 905209 INFO  (qtp2053162384-7435) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/oba path=/select params={q=*:*&distrib=false&fl=id,a_i1&group.limit=10&sort=a_i1+asc,+id+asc&rows=100&wt=javabin&version=2&group.field=a_i1&group=true} hits=0 status=0 QTime=6 
   [junit4]   2> 905281 INFO  (qtp1400273651-7452) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/oba path=/select params={group.distributed.first=true&distrib=false&fl=id&group.limit=10&shards.purpose=2048&start=0&sort=a_i1+asc,+id+asc&shard.url=https://127.0.0.1:46822/oba/collection1&rows=100&version=2&q=*:*&NOW=1433920774686&isShard=true&wt=javabin&group.field=a_i1&group=true} status=0 QTime=11 
   [junit4]   2> 905287 INFO  (qtp1400273651-7453) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/oba path=/select params={q=*:*&shards=https://127.0.0.1:46822/oba/collection1&fl=id,a_i1&group.limit=10&sort=a_i1+asc,+id+asc&rows=100&wt=javabin&version=2&group.field=a_i1&group=true} status=0 QTime=75 
   [junit4]   2> 905290 INFO  (qtp2053162384-7433) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/oba path=/select params={q=*:*&distrib=false&hl=true&fl=id,a_i1&group.limit=10&sort=a_i1+asc,+id+asc&hl.fl=a_t&rows=100&wt=javabin&version=2&group.field=a_i1&group=true} hits=0 status=0 QTime=1 
   [junit4]   2> 905293 INFO  (qtp1400273651-7452) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/oba path=/select params={group.distributed.first=true&distrib=false&hl=false&fl=id&group.limit=10&shards.purpose=2048&start=0&sort=a_i1+asc,+id+asc&shard.url=https://127.0.0.1:46822/oba/collection1&rows=100&version=2&q=*:*&NOW=1433920774765&isShard=true&hl.fl=a_t&wt=javabin&group.field=a_i1&group=true} status=0 QTime=0 
   [junit4]   2> 905294 INFO  (qtp1400273651-7451) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/oba path=/select params={q=*:*&shards=https://127.0.0.1:46822/oba/collection1&hl=true&fl=id,a_i1&group.limit=10&sort=a_i1+asc,+id+asc&hl.fl=a_t&rows=100&wt=javabin&version=2&group.field=a_i1&group=true} status=0 QTime=3 
   [junit4]   2> 905296 INFO  (qtp2053162384-7434) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/oba path=/select params={q=*:*&facet.field=a_t&distrib=false&fl=id,a_i1&group.limit=10&sort=a_i1+asc,+id+asc&rows=100&facet=true&wt=javabin&version=2&group.field=a_i1&group=true} hits=0 status=0 QTime=0 
   [junit4]   2> 905313 INFO  (qtp1400273651-7452) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/oba path=/select params={facet.field=a_t&group.distributed.first=true&distrib=false&fl=id&group.limit=10&shards.purpose=2048&start=0&sort=a_i1+asc,+id+asc&shard.url=https://127.0.0.1:46822/oba/collection1|[ff01::083]:33332/oba|[ff01::114]:33332/oba&rows=100&version=2&q=*:*&NOW=1433920774772&isShard=true&facet=false&wt=javabin&group.field=a_i1&group=true} status=0 QTime=0 
   [junit4]   2> 905323 INFO  (qtp1400273651-7451) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/oba path=/select params={facet.field=a_t&distrib=false&fl=id&group.limit=10&shards.purpose=20&start=0&f.a_t.facet.mincount=0&sort=a_i1+asc,+id+asc&shard.url=https://127.0.0.1:46822/oba/collection1|[ff01::083]:33332/oba|[ff01::114]:33332/oba&rows=100&version=2&group.distributed.second=true&q=*:*&f.a_t.facet.limit=160&NOW=1433920774772&isShard=true&facet=true&wt=javabin&group.field=a_i1&group=true} status=0 QTime=6 
   [junit4]   2> 905327 INFO  (qtp1400273651-7453) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/oba path=/select params={q=*:*&shards=https://127.0.0.1:46822/oba/collection1|[ff01::083]:33332/oba|[ff01::114]:33332/oba&facet.field=a_t&fl=id,a_i1&group.limit=10&sort=a_i1+asc,+id+asc&rows=100&facet=true&wt=javabin&version=2&group.field=a_i1&group=true} status=0 QTime=29 
   [junit4]   2> 905330 INFO  (qtp2053162384-7435) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/oba path=/select params={q=*:*&distrib=false&stats=true&fl=id,a_i1&group.limit=10&sort=a_i1+asc,+id+asc&rows=100&wt=javabin&version=2&group.field=a_i1&group=true&stats.field=a_i1} hits=0 status=0 QTime=1 
   [junit4]   2> 905355 INFO  (qtp1400273651-7451) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/oba path=/select params={group.distributed.first=true&distrib=false&fl=id&group.limit=10&shards.purpose=2048&start=0&sort=a_i1+asc,+id+asc&shard.url=https://127.0.0.1:46822/oba/collection1|[ff01::213]:33332/oba&rows=100&version=2&q=*:*&stats=false&NOW=1433920774805&isShard=true&wt=javabin&group.field=a_i1&group=true&stats.field=a_i1} status=0 QTime=0 
   [junit4]   2> 905357 INFO  (qtp1400273651-7453) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/oba path=/select params={distrib=false&fl=id&group.limit=10&shards.purpose=516&start=0&sort=a_i1+asc,+id+asc&shard.url=https://127.0.0.1:46822/oba/collection1|[ff01::213]:33332/oba&rows=100&version=2&group.distributed.second=true&q=*:*&stats=true&NOW=1433920774805&isShard=true&wt=javabin&group.field=a_i1&group=true&stats.field=a_i1} status=0 QTime=0 
   [junit4]   2> 905357 INFO  (qtp1400273651-7452) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/oba path=/select params={q=*:*&shards=https://127.0.0.1:46822/oba/collection1|[ff01::213]:33332/oba&stats=true&fl=id,a_i1&group.limit=10&sort=a_i1+asc,+id+asc&rows=100&wt=javabin&version=2&group.field=a_i1&group=true&stats.field=a_i1} status=0 QTime=26 
   [junit4]   2> 905382 INFO  (qtp2053162384-7433) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/oba path=/select params={distrib=false&qt=spellCheckCompRH&fl=id,a_i1&group.limit=10&sort=a_i1+asc,+id+asc&rows=100&version=2&spellcheck.build=true&q=kings&spellcheck=true&wt=javabin&group.field=a_i1&group=true} hits=0 status=0 QTime=23 
   [junit4]   2> 905419 INFO  (qtp1400273651-7453) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/oba path=/select params={group.distributed.first=true&distrib=false&qt=spellCheckCompRH&fl=id&group.limit=10&shards.purpose=2048&start=0&sort=a_i1+asc,+id+asc&shard.url=https://127.0.0.1:46822/oba/collection1&rows=100&version=2&spellcheck.build=true&spellcheck.extendedResults=false&q=kings&spellcheck=true&NOW=1433920774859&spellcheck.onlyMorePopular=false&isShard=true&spellcheck.count=5&wt=javabin&group.field=a_i1&group=true} status=0 QTime=20 
   [junit4]   2> 905421 INFO  (qtp1400273651-7451) [    x:collection1] o.a.s.h.c.SpellCheckComponent https://127.0.0.1:46822/oba/collection1 {suggestions={kings={numFound=0,startOffset=0,endOffset=5,suggestion=[]}},originalTerms=[kings]}
   [junit4]   2> 905422 INFO  (qtp1400273651-7451) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/oba path=/select params={qt=spellCheckCompRH&fl=id,a_i1&group.limit=10&sort=a_i1+asc,+id+asc&rows=100&version=2&spellcheck.build=true&q=kings&shards=https://127.0.0.1:46822/oba/collection1&spellcheck=true&wt=javabin&group.field=a_i1&group=true} status=0 QTime=37 
   [junit4]   2> 905426 INFO  (qtp2053162384-7434) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/oba path=/select params={facet.field=a_t&distrib=false&fl=id,a_i1&group.limit=10&fq=a_s:a&sort=a_i1+asc,+id+asc&rows=100&version=2&q=*:*&group.truncate=true&facet=true&wt=javabin&group.field=a_i1&group=true} hits=0 status=0 QTime=2 
   [junit4]   2> 905430 INFO  (qtp1400273651-7453) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/oba path=/select params={facet.field=a_t&group.distributed.first=true&distrib=false&fl=id&group.limit=10&shards.purpose=2048&start=0&fq=a_s:a&sort=a_i1+asc,+id+asc&shard.url=https://127.0.0.1:46822/oba/collection1&rows=100&version=2&q=*:*&group.truncate=true&NOW=1433920774902&isShard=true&facet=false&wt=javabin&group.field=a_i1&group=true} status=0 QTime=1 
   [junit4]   2> 905432 INFO  (qtp1400273651-7451) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/oba path=/select params={facet.field=a_t&distrib=false&fl=id&group.limit=10&shards.purpose=20&start=0&f.a_t.facet.mincount=0&fq=a_s:a&sort=a_i1+asc,+id+asc&shard.url=https://127.0.0.1:46822/oba/collection1&rows=100&version=2&group.distributed.second=true&q=*:*&f.a_t.facet.limit=160&group.truncate=true&NOW=1433920774902&isShard=true&facet=true&wt=javabin&group.field=a_i1&group=true} status=0 QTime=0 
   [junit4]   2> 905432 INFO  (qtp1400273651-7452) [    x:collection1] o.a.s.c.S.Request [collection1] webapp=/oba path=/select params={facet.field=a_t&fl=id,a_i1&group.limit=10&fq=a_s:a&sort=a_i1+asc,+id+asc&rows=100&version=2&q=*:*&shards=https://127.0.0.1:46822/oba/collection1&group.truncate=true&facet=true&wt=javabin&group.field=a_i1&group=true} status=0 QTime=4 
   [junit4]   2> 905437 INFO  (qtp2053162384-7435) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[1 (1503574910470979584)]} 0 3
   [junit4]   2> 905441 INFO  (qtp1400273651-7453) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[1 (1503574910475173888)]} 0 2
   [junit4]   2> 905443 INFO  (qtp2053162384-7433) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[2 (1503574910479368192)]} 0 0
   [junit4]   2> 905446 INFO  (qtp1400273651-7451) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[2 (1503574910481465344)]} 0 1
   [junit4]   2> 905449 INFO  (qtp2053162384-7434) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[3 (1503574910483562496)]} 0 2
   [junit4]   2> 905451 INFO  (qtp1400273651-7452) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[3 (1503574910486708224)]} 0 1
   [junit4]   2> 905453 INFO  (qtp2053162384-7435) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[4 (1503574910489853952)]} 0 0
   [junit4]   2> 905456 INFO  (qtp1400273651-7453) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[4 (1503574910491951104)]} 0 1
   [junit4]   2> 905458 INFO  (qtp2053162384-7433) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[5 (1503574910494048256)]} 0 1
   [junit4]   2> 905460 INFO  (qtp1400273651-7451) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[5 (1503574910497193984)]} 0 1
   [junit4]   2> 905462 INFO  (qtp2053162384-7434) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[6 (1503574910499291136)]} 0 0
   [junit4]   2> 905464 INFO  (qtp1400273651-7452) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[6 (1503574910501388288)]} 0 0
   [junit4]   2> 905466 INFO  (qtp2053162384-7435) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[7 (1503574910503485440)]} 0 0
   [junit4]   2> 905474 INFO  (qtp1400273651-7453) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[7 (1503574910505582592)]} 0 6
   [junit4]   2> 905477 INFO  (qtp2053162384-7433) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[8 (1503574910513971200)]} 0 1
   [junit4]   2> 905479 INFO  (qtp1400273651-7451) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[8 (1503574910516068352)]} 0 1
   [junit4]   2> 905481 INFO  (qtp2053162384-7434) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[9 (1503574910519214080)]} 0 0
   [junit4]   2> 905484 INFO  (qtp1400273651-7452) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[9 (1503574910521311232)]} 0 1
   [junit4]   2> 905486 INFO  (qtp2053162384-7435) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[10 (1503574910524456960)]} 0 1
   [junit4]   2> 905494 INFO  (qtp1400273651-7453) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[10 (1503574910525505536)]} 0 7
   [junit4]   2> 905497 INFO  (qtp2053162384-7433) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[11 (1503574910534942720)]} 0 1
   [junit4]   2> 905505 INFO  (qtp1400273651-7451) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[11 (1503574910537039872)]} 0 7
   [junit4]   2> 905507 INFO  (qtp2053162384-7434) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[12 (1503574910546477056)]} 0 0
   [junit4]   2> 905515 INFO  (qtp1400273651-7452) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[12 (1503574910547525632)]} 0 7
   [junit4]   2> 905517 INFO  (qtp2053162384-7435) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[14 (1503574910556962816)]} 0 0
   [junit4]   2> 905519 INFO  (qtp1400273651-7453) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[14 (1503574910559059968)]} 0 0
   [junit4]   2> 905521 INFO  (qtp2053162384-7433) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[15 (1503574910561157120)]} 0 0
   [junit4]   2> 905523 INFO  (qtp1400273651-7451) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[15 (1503574910563254272)]} 0 0
   [junit4]   2> 905535 INFO  (qtp2053162384-7434) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[16 (1503574910565351424)]} 0 10
   [junit4]   2> 905537 INFO  (qtp1400273651-7452) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[16 (1503574910577934336)]} 0 0
   [junit4]   2> 905540 INFO  (qtp2053162384-7435) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[17 (1503574910580031488)]} 0 1
   [junit4]   2> 905543 INFO  (qtp1400273651-7453) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[17 (1503574910583177216)]} 0 1
   [junit4]   2> 905546 INFO  (qtp2053162384-7433) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[18 (1503574910586322944)]} 0 2
   [junit4]   2> 905549 INFO  (qtp1400273651-7451) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[18 (1503574910589468672)]} 0 1
   [junit4]   2> 905551 INFO  (qtp2053162384-7434) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[19 (1503574910592614400)]} 0 0
   [junit4]   2> 905553 INFO  (qtp1400273651-7452) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[19 (1503574910594711552)]} 0 0
   [junit4]   2> 905556 INFO  (qtp2053162384-7435) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[20 (1503574910596808704)]} 0 1
   [junit4]   2> 905558 INFO  (qtp1400273651-7453) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[20 (1503574910599954432)]} 0 1
   [junit4]   2> 905560 INFO  (qtp2053162384-7433) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[21 (1503574910602051584)]} 0 0
   [junit4]   2> 905562 INFO  (qtp1400273651-7451) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[21 (1503574910604148736)]} 0 0
   [junit4]   2> 905564 INFO  (qtp2053162384-7434) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[22 (1503574910606245888)]} 0 0
   [junit4]   2> 905567 INFO  (qtp1400273651-7452) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[22 (1503574910608343040)]} 0 1
   [junit4]   2> 905569 INFO  (qtp2053162384-7435) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[23 (1503574910611488768)]} 0 0
   [junit4]   2> 905578 INFO  (qtp1400273651-7453) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[23 (1503574910619877376)]} 0 1
   [junit4]   2> 905580 INFO  (qtp2053162384-7433) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[100 (1503574910623023104)]} 0 0
   [junit4]   2> 905582 INFO  (qtp1400273651-7451) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[100 (1503574910625120256)]} 0 0
   [junit4]   2> 905584 INFO  (qtp2053162384-7434) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[101 (1503574910627217408)]} 0 0
   [junit4]   2> 905587 INFO  (qtp1400273651-7452) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[101 (1503574910629314560)]} 0 1
   [junit4]   2> 905589 INFO  (qtp2053162384-7435) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[102 (1503574910632460288)]} 0 0
   [junit4]   2> 905591 INFO  (qtp1400273651-7453) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[102 (1503574910634557440)]} 0 0
   [junit4]   2> 905593 INFO  (qtp2053162384-7433) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[103 (1503574910636654592)]} 0 0
   [junit4]   2> 905595 INFO  (qtp1400273651-7451) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[103 (1503574910638751744)]} 0 0
   [junit4]   2> 905597 INFO  (qtp2053162384-7434) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[104 (1503574910640848896)]} 0 0
   [junit4]   2> 905599 INFO  (qtp1400273651-7452) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update params={wt=javabin&version=2} {add=[104 (1503574910642946048)]} 0 0
   [junit4]   2> 905601 INFO  (qtp2053162384-7435) [    x:collection1] o.a.s.u.p.LogUpdateProcessor [collection1] webapp=/oba path=/update 

[...truncated too long message...]

penCloseCoreStressTest, LeaderElectionIntegrationTest, ZkSolrClientTest, TestRandomDVFaceting, TestDistributedGrouping]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDistributedGrouping -Dtests.seed=9C7CF3E7DBC4FE77 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=mt -Dtests.timezone=Australia/LHI -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s J1 | TestDistributedGrouping (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.TestDistributedGrouping: 
   [junit4]    >    1) Thread[id=7553, name=searcherExecutor-3689-thread-1, state=WAITING, group=TGRP-TestDistributedGrouping]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    2) Thread[id=7545, name=qtp70567688-7545, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
   [junit4]    >         at java.util.WeakHashMap.get(WeakHashMap.java:403)
   [junit4]    >         at org.apache.solr.servlet.cache.HttpCacheHeaderUtil.calcEtag(HttpCacheHeaderUtil.java:101)
   [junit4]    >         at org.apache.solr.servlet.cache.HttpCacheHeaderUtil.doCacheHeaderValidation(HttpCacheHeaderUtil.java:219)
   [junit4]    >         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:444)
   [junit4]    >         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
   [junit4]    >         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
   [junit4]    >         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
   [junit4]    >         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:106)
   [junit4]    >         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
   [junit4]    >         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)
   [junit4]    >         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:364)
   [junit4]    >         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
   [junit4]    >         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
   [junit4]    >         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
   [junit4]    >         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
   [junit4]    >         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
   [junit4]    >         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]    >         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
   [junit4]    >         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]    >         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
   [junit4]    >         at org.eclipse.jetty.server.Server.handle(Server.java:497)
   [junit4]    >         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
   [junit4]    >         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
   [junit4]    >         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9C7CF3E7DBC4FE77]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=7553, name=searcherExecutor-3689-thread-1, state=WAITING, group=TGRP-TestDistributedGrouping]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    2) Thread[id=7545, name=qtp70567688-7545, state=RUNNABLE, group=TGRP-TestDistributedGrouping]
   [junit4]    >         at java.util.WeakHashMap.get(WeakHashMap.java:403)
   [junit4]    >         at org.apache.solr.servlet.cache.HttpCacheHeaderUtil.calcEtag(HttpCacheHeaderUtil.java:101)
   [junit4]    >         at org.apache.solr.servlet.cache.HttpCacheHeaderUtil.doCacheHeaderValidation(HttpCacheHeaderUtil.java:219)
   [junit4]    >         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:444)
   [junit4]    >         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
   [junit4]    >         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
   [junit4]    >         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
   [junit4]    >         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:106)
   [junit4]    >         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
   [junit4]    >         at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)
   [junit4]    >         at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:364)
   [junit4]    >         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
   [junit4]    >         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
   [junit4]    >         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
   [junit4]    >         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
   [junit4]    >         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
   [junit4]    >         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]    >         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
   [junit4]    >         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]    >         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
   [junit4]    >         at org.eclipse.jetty.server.Server.handle(Server.java:497)
   [junit4]    >         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
   [junit4]    >         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
   [junit4]    >         at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9C7CF3E7DBC4FE77]:0)
   [junit4] Completed [381/500] on J1 in 156.17s, 1 test, 3 errors <<< FAILURES!

[...truncated 389 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:536: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:484: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:61: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-5.x-Linux/extra-targets.xml:39: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build.xml:230: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/common-build.xml:511: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/common-build.xml:1438: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/common-build.xml:993: There were test failures: 500 suites, 1978 tests, 2 suite-level errors, 1 error, 348 ignored (25 assumptions)

Total time: 40 minutes 31 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any