You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2017/12/08 03:48:15 UTC
[JENKINS] Lucene-Solr-7.x-Linux (64bit/jdk1.8.0_144) - Build # 956
- Still Failing!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/956/
Java: 64bit/jdk1.8.0_144 -XX:-UseCompressedOops -XX:+UseParallelGC
4 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.core.TestLazyCores
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores: 1) Thread[id=20473, name=searcherExecutor-5945-thread-1, state=WAITING, group=TGRP-TestLazyCores] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores:
1) Thread[id=20473, name=searcherExecutor-5945-thread-1, state=WAITING, group=TGRP-TestLazyCores]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([52E42445FD98FFF3]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.core.TestLazyCores
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=20473, name=searcherExecutor-5945-thread-1, state=WAITING, group=TGRP-TestLazyCores] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=20473, name=searcherExecutor-5945-thread-1, state=WAITING, group=TGRP-TestLazyCores]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([52E42445FD98FFF3]:0)
FAILED: org.apache.solr.core.TestLazyCores.testNoCommit
Error Message:
Exception during query
Stack Trace:
java.lang.RuntimeException: Exception during query
at __randomizedtesting.SeedInfo.seed([52E42445FD98FFF3:8D84859436BF9C56]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:901)
at org.apache.solr.core.TestLazyCores.check10(TestLazyCores.java:847)
at org.apache.solr.core.TestLazyCores.testNoCommit(TestLazyCores.java:829)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound='10']
xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">1</int><lst name="params"><str name="q">*:*</str></lst></lst><result name="response" numFound="0" start="0"></result>
</response>
request was:q=*:*
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:894)
... 41 more
FAILED: org.apache.solr.handler.TestSolrConfigHandlerConcurrent.test
Error Message:
Timeout occured while waiting response from server at: https://127.0.0.1:34733
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:34733
at __randomizedtesting.SeedInfo.seed([52E42445FD98FFF3:DAB01B9F5364920B]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1103)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:883)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:314)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:991)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:940)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
... 43 more
Build Log:
[...truncated 12506 lines...]
[junit4] Suite: org.apache.solr.handler.TestSolrConfigHandlerConcurrent
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.handler.TestSolrConfigHandlerConcurrent_52E42445FD98FFF3-001/init-core-data-001
[junit4] 2> 895513 WARN (SUITE-TestSolrConfigHandlerConcurrent-seed#[52E42445FD98FFF3]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
[junit4] 2> 895513 INFO (SUITE-TestSolrConfigHandlerConcurrent-seed#[52E42445FD98FFF3]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 895514 INFO (SUITE-TestSolrConfigHandlerConcurrent-seed#[52E42445FD98FFF3]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 895515 INFO (SUITE-TestSolrConfigHandlerConcurrent-seed#[52E42445FD98FFF3]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 895515 INFO (SUITE-TestSolrConfigHandlerConcurrent-seed#[52E42445FD98FFF3]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 895516 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 895516 INFO (Thread-1708) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 895516 INFO (Thread-1708) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 895517 ERROR (Thread-1708) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 895616 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.ZkTestServer start zk server on port:35525
[junit4] 2> 895617 INFO (zkConnectionManagerCallback-1476-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 895619 INFO (zkConnectionManagerCallback-1478-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 895621 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 895621 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 895622 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 895622 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 895622 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 895623 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 895623 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 895623 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 895624 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 895624 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 895624 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 895625 INFO (zkConnectionManagerCallback-1481-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 895625 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 895677 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 895679 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3fc03c5f{/,null,AVAILABLE}
[junit4] 2> 895680 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5619d543{SSL,[ssl, http/1.1]}{127.0.0.1:34733}
[junit4] 2> 895680 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.e.j.s.Server Started @898029ms
[junit4] 2> 895680 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.handler.TestSolrConfigHandlerConcurrent_52E42445FD98FFF3-001/tempDir-001/control/data, replicaType=NRT, hostContext=/, hostPort=34733, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.handler.TestSolrConfigHandlerConcurrent_52E42445FD98FFF3-001/control-001/cores}
[junit4] 2> 895681 ERROR (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 895681 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 895681 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 895681 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 895681 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-08T03:22:35.316Z
[junit4] 2> 895685 INFO (zkConnectionManagerCallback-1483-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 895686 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 895686 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.handler.TestSolrConfigHandlerConcurrent_52E42445FD98FFF3-001/control-001/solr.xml
[junit4] 2> 895688 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 895688 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 895689 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@39406884, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 895691 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35525/solr
[junit4] 2> 895691 INFO (zkConnectionManagerCallback-1487-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 895692 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x1603425690f0004, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 895693 INFO (zkConnectionManagerCallback-1489-thread-1-processing-n:127.0.0.1:34733_) [n:127.0.0.1:34733_ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 895715 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [n:127.0.0.1:34733_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 895715 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [n:127.0.0.1:34733_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34733_
[junit4] 2> 895715 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [n:127.0.0.1:34733_ ] o.a.s.c.Overseer Overseer (id=99136527083175941-127.0.0.1:34733_-n_0000000000) starting
[junit4] 2> 895721 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [n:127.0.0.1:34733_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34733_
[junit4] 2> 895721 INFO (zkCallback-1488-thread-1-processing-n:127.0.0.1:34733_) [n:127.0.0.1:34733_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 895795 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [n:127.0.0.1:34733_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@39406884
[junit4] 2> 895799 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [n:127.0.0.1:34733_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@39406884
[junit4] 2> 895799 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [n:127.0.0.1:34733_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@39406884
[junit4] 2> 895800 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [n:127.0.0.1:34733_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.handler.TestSolrConfigHandlerConcurrent_52E42445FD98FFF3-001/control-001/cores
[junit4] 2> 895810 INFO (zkConnectionManagerCallback-1495-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 895811 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 895811 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35525/solr ready
[junit4] 2> 895828 INFO (qtp1394976278-6852) [n:127.0.0.1:34733_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:34733_&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 895829 INFO (OverseerThreadFactory-2623-thread-1-processing-n:127.0.0.1:34733_) [n:127.0.0.1:34733_ ] o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 895934 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 895934 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 896037 INFO (zkCallback-1488-thread-1-processing-n:127.0.0.1:34733_) [n:127.0.0.1:34733_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 896951 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 896968 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 897062 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 897072 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 897072 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@39406884
[junit4] 2> 897073 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 897073 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.handler.TestSolrConfigHandlerConcurrent_52E42445FD98FFF3-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.handler.TestSolrConfigHandlerConcurrent_52E42445FD98FFF3-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 897074 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=29, maxMergeAtOnceExplicit=35, maxMergedSegmentMB=12.5068359375, floorSegmentMB=1.0888671875, forceMergeDeletesPctAllowed=7.279987561596551, segmentsPerTier=14.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.29920422115360057
[junit4] 2> 897078 WARN (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 897107 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 897107 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 897108 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 897108 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 897109 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=17, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7472190767096655]
[junit4] 2> 897109 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@424e4b77[control_collection_shard1_replica_n1] main]
[junit4] 2> 897110 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 897113 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 897113 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 897113 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1586184435005390848
[junit4] 2> 897114 INFO (searcherExecutor-2626-thread-1-processing-n:127.0.0.1:34733_ x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@424e4b77[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 897116 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 897116 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 897116 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:34733/control_collection_shard1_replica_n1/
[junit4] 2> 897116 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 897116 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:34733/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 897116 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 897117 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:34733/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 955854 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x1603425690f0006, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 955854 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.ChaosMonkey monkey: stop jetty! 34733
[junit4] 2> 955854 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1379448781
[junit4] 2> 955854 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 955854 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@11ce32ae: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@21c6ee6f
[junit4] 2> 955857 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 955857 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@59576666: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@515cdf3b
[junit4] 2> 955859 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 955859 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@688a6f41: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@999cd4a
[junit4] 2> 955859 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:34733_
[junit4] 2> 955860 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 955860 INFO (zkCallback-1488-thread-2-processing-n:127.0.0.1:34733_) [n:127.0.0.1:34733_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 955860 INFO (coreCloseExecutor-2631-thread-1) [n:127.0.0.1:34733_ c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@6cddb963
[junit4] 2> 955860 INFO (coreCloseExecutor-2631-thread-1) [n:127.0.0.1:34733_ c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.control_collection.shard1.replica_n1, tag=1826470243
[junit4] 2> 955860 INFO (coreCloseExecutor-2631-thread-1) [n:127.0.0.1:34733_ c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@24fe680b: rootName = null, domain = solr.core.control_collection.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.control_collection.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@662d820b
[junit4] 2> 955864 INFO (coreCloseExecutor-2631-thread-1) [n:127.0.0.1:34733_ c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.control_collection.shard1.leader, tag=1826470243
[junit4] 2> 966746 INFO (ScheduledTrigger-2621-thread-1) [n:127.0.0.1:34733_ ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 966747 INFO (AutoscalingActionExecutor-2622-thread-1-processing-n:127.0.0.1:34733_) [n:127.0.0.1:34733_ ] o.a.s.c.a.ExecutePlanAction No operations to execute for event: {
[junit4] 2> "id":"14fe33be17bb0280Teavc84im6hlrxl6ailb0skwq7",
[junit4] 2> "source":".auto_add_replicas",
[junit4] 2> "eventTime":1512703416378000000,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1512703416378000000],
[junit4] 2> "_enqueue_time_":1512703426380000000,
[junit4] 2> "nodeNames":["127.0.0.1:34733_"]}}
[junit4] 2> 1075829 ERROR (qtp1394976278-6852) [n:127.0.0.1:34733_ ] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: create the collection time out:180s
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:314)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:246)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:224)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:735)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:716)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:497)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 1075830 INFO (qtp1394976278-6852) [n:127.0.0.1:34733_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:34733_&wt=javabin&version=2} status=500 QTime=180001
[junit4] 2> 1075830 ERROR (qtp1394976278-6852) [n:127.0.0.1:34733_ ] o.a.s.s.HttpSolrCall null:org.apache.solr.common.SolrException: create the collection time out:180s
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:314)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:246)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:224)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:735)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:716)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:497)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 1496025 ERROR (OverseerThreadFactory-2623-thread-1-processing-n:127.0.0.1:34733_) [n:127.0.0.1:34733_ ] o.a.s.c.OverseerCollectionMessageHandler Error from shard: https://127.0.0.1:34733
[junit4] 2> org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:34733
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
[junit4] 2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
[junit4] 2> at org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:172)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: java.net.SocketTimeoutException: Read timed out
[junit4] 2> at java.net.SocketInputStream.socketRead0(Native Method)
[junit4] 2> at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
[junit4] 2> at java.net.SocketInputStream.read(SocketInputStream.java:171)
[junit4] 2> at java.net.SocketInputStream.read(SocketInputStream.java:141)
[junit4] 2> at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
[junit4] 2> at sun.security.ssl.InputRecord.read(InputRecord.java:503)
[junit4] 2> at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983)
[junit4] 2> at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:940)
[junit4] 2> at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
[junit4] 2> at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
[junit4] 2> at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
[junit4] 2> at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
[junit4] 2> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
[junit4] 2> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
[junit4] 2> at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
[junit4] 2> at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
[junit4] 2> at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
[junit4] 2> at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
[junit4] 2> at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
[junit4] 2> at org.apache.solr.util.stats.InstrumentedHttpRequestExecutor.execute(InstrumentedHttpRequestExecutor.java:118)
[junit4] 2> at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
[junit4] 2> at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
[junit4] 2> at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
[junit4] 2> at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
[junit4] 2> at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
[junit4] 2> ... 12 more
[junit4] 2> 1496025 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.Overseer Overseer (id=99136527083175941-127.0.0.1:34733_-n_0000000000) closing
[junit4] 2> 1496026 INFO (OverseerStateUpdate-99136527083175941-127.0.0.1:34733_-n_0000000000) [n:127.0.0.1:34733_ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:34733_
[junit4] 2> 1496026 ERROR (OverseerThreadFactory-2623-thread-1-processing-n:127.0.0.1:34733_) [n:127.0.0.1:34733_ ] o.a.s.c.OverseerCollectionMessageHandler Cleaning up collection [control_collection].
[junit4] 2> 1496026 INFO (OverseerThreadFactory-2623-thread-1-processing-n:127.0.0.1:34733_) [n:127.0.0.1:34733_ ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
[junit4] 2> 1526055 INFO (zkCallback-1488-thread-5-processing-n:127.0.0.1:34733_) [n:127.0.0.1:34733_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [0])
[junit4] 2> 1526055 ERROR (OverseerThreadFactory-2623-thread-1-processing-n:127.0.0.1:34733_) [n:127.0.0.1:34733_ ] o.a.s.c.OverseerCollectionMessageHandler Collection: control_collection operation: create failed:org.apache.solr.common.SolrException: Could not fully remove collection: control_collection
[junit4] 2> at org.apache.solr.cloud.DeleteCollectionCmd.call(DeleteCollectionCmd.java:109)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionMessageHandler.cleanupCollection(OverseerCollectionMessageHandler.java:679)
[junit4] 2> at org.apache.solr.cloud.CreateCollectionCmd.call(CreateCollectionCmd.java:302)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:233)
[junit4] 2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:464)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 1526056 INFO (OverseerThreadFactory-2623-thread-1-processing-n:127.0.0.1:34733_) [n:127.0.0.1:34733_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1526057 WARN (OverseerAutoScalingTriggerThread-99136527083175941-127.0.0.1:34733_-n_0000000000) [n:127.0.0.1:34733_ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1526058 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@5619d543{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 1526058 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3fc03c5f{/,null,UNAVAILABLE}
[junit4] 2> 1526090 ERROR (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController Error getting leader from zk
[junit4] 2> org.apache.solr.common.SolrException: No registered leader was found after waiting for 1220000ms , collection: control_collection slice: shard1 saw state=null with live_nodes=[]
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:772)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:737)
[junit4] 2> at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1178)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1079)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1014)
[junit4] 2> at org.apache.solr.core.ZkContainer.lambda$registerInZk$0(ZkContainer.java:181)
[junit4] 2> at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:208)
[junit4] 2> at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:886)
[junit4] 2> at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1047)
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:948)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:735)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:716)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:497)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1526091 ERROR (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkContainer :org.apache.solr.common.SolrException: Error getting leader from zk for shard shard1
[junit4] 2> at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1207)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1079)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1014)
[junit4] 2> at org.apache.solr.core.ZkContainer.lambda$registerInZk$0(ZkContainer.java:181)
[junit4] 2> at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:208)
[junit4] 2> at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:886)
[junit4] 2> at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1047)
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:948)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:735)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:716)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:497)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: org.apache.solr.common.SolrException: No registered leader was found after waiting for 1220000ms , collection: control_collection slice: shard1 saw state=null with live_nodes=[]
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:772)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:737)
[junit4] 2> at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1178)
[junit4] 2> ... 44 more
[junit4] 2>
[junit4] 2> 1526101 INFO (qtp1394976278-6858) [n:127.0.0.1:34733_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=630167
[junit4] 2> 1526102 ERROR (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1526102 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:35525 35525
[junit4] 2> 1536728 INFO (Thread-1708) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:35525 35525
[junit4] 2> 1536728 WARN (Thread-1708) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/aliases.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/clusterstate.json
[junit4] 2> 2 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/live_nodes
[junit4] 2> 2 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSolrConfigHandlerConcurrent -Dtests.method=test -Dtests.seed=52E42445FD98FFF3 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es-CO -Dtests.timezone=PLT -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 641s J2 | TestSolrConfigHandlerConcurrent.test <<<
[junit4] > Throwable #1: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:34733
[junit4] > at __randomizedtesting.SeedInfo.seed([52E42445FD98FFF3:DAB01B9F5364920B]:0)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1103)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:883)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
[junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
[junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:314)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:991)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > Caused by: java.net.SocketTimeoutException: Read timed out
[junit4] > at java.net.SocketInputStream.socketRead0(Native Method)
[junit4] > at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
[junit4] > at java.net.SocketInputStream.read(SocketInputStream.java:171)
[junit4] > at java.net.SocketInputStream.read(SocketInputStream.java:141)
[junit4] > at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
[junit4] > at sun.security.ssl.InputRecord.read(InputRecord.java:503)
[junit4] > at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983)
[junit4] > at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:940)
[junit4] > at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
[junit4] > at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
[junit4] > at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
[junit4] > at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
[junit4] > at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
[junit4] > at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
[junit4] > at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
[junit4] > at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
[junit4] > at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
[junit4] > at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
[junit4] > at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
[junit4] > at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
[junit4] > at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
[junit4] > at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
[junit4] > at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
[junit4] > at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
[junit4] > at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
[junit4] > at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
[junit4] > ... 43 more
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.handler.TestSolrConfigHandlerConcurrent_52E42445FD98FFF3-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=582, maxMBSortInHeap=6.56271208006368, sim=RandomSimilarity(queryNorm=false): {}, locale=es-CO, timezone=PLT
[junit4] 2> NOTE: Linux 4.10.0-40-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=8,threads=1,free=156953272,total=530579456
[junit4] 2> NOTE: All tests run in this JVM: [ConfigSetsAPITest, TestPullReplica, TestSchemalessBufferedUpdates, CloudMLTQParserTest, TestHalfAndHalfDocValues, AddBlockUpdateTest, TestLegacyField, DefaultValueUpdateProcessorTest, TestSolrCloudWithHadoopAuthPlugin, VersionInfoTest, DistributedFacetPivotSmallTest, WordBreakSolrSpellCheckerTest, LeaderElectionContextKeyTest, ReturnFieldsTest, UnloadDistributedZkTest, TestLeaderInitiatedRecoveryThread, TestGroupingSearch, ScriptEngineTest, TestNoOpRegenerator, TestExactSharedStatsCache, TestRequestForwarding, SpellPossibilityIteratorTest, TestRestManager, CoreAdminHandlerTest, SearchRateTriggerTest, TestCollectionAPIs, TestPivotHelperCode, HLLUtilTest, LeaderElectionTest, TestMiniSolrCloudClusterSSL, HdfsRecoveryZkTest, TestSolrFieldCacheBean, BasicAuthIntegrationTest, TestZkChroot, TestSha256AuthenticationProvider, CoreAdminRequestStatusTest, SolrShardReporterTest, TestMaxScoreQueryParser, TestOrdValues, TestDistribDocBasedVersion, QueryElevationComponentTest, CdcrBootstrapTest, BigEndianAscendingWordDeserializerTest, ClassificationUpdateProcessorTest, TestFieldCacheReopen, AutoAddReplicasIntegrationTest, TestExportWriter, TestSubQueryTransformerDistrib, TestCSVResponseWriter, AlternateDirectoryTest, RecoveryZkTest, DistributedDebugComponentTest, TestHashPartitioner, InfixSuggestersTest, TestInPlaceUpdatesDistrib, IndexSchemaTest, TestQuerySenderListener, TestSolrConfigHandlerConcurrent]
[junit4] Completed [302/751 (1!)] on J2 in 641.24s, 1 test, 1 error <<< FAILURES!
[...truncated 377 lines...]
[junit4] Suite: org.apache.solr.core.TestLazyCores
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestLazyCores_52E42445FD98FFF3-001/init-core-data-001
[junit4] 2> 1856818 WARN (SUITE-TestLazyCores-seed#[52E42445FD98FFF3]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=39 numCloses=39
[junit4] 2> 1856818 INFO (SUITE-TestLazyCores-seed#[52E42445FD98FFF3]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1857279 INFO (SUITE-TestLazyCores-seed#[52E42445FD98FFF3]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 1857279 INFO (SUITE-TestLazyCores-seed#[52E42445FD98FFF3]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 1857281 INFO (TEST-TestLazyCores.testBadConfigsGenerateErrors-seed#[52E42445FD98FFF3]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testBadConfigsGenerateErrors
[junit4] 2> 1857287 INFO (TEST-TestLazyCores.testBadConfigsGenerateErrors-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@4473d4be, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1857306 INFO (TEST-TestLazyCores.testBadConfigsGenerateErrors-seed#[52E42445FD98FFF3]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4473d4be
[junit4] 2> 1857311 INFO (TEST-TestLazyCores.testBadConfigsGenerateErrors-seed#[52E42445FD98FFF3]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4473d4be
[junit4] 2> 1857311 INFO (TEST-TestLazyCores.testBadConfigsGenerateErrors-seed#[52E42445FD98FFF3]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4473d4be
[junit4] 2> 1857313 INFO (TEST-TestLazyCores.testBadConfigsGenerateErrors-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.CorePropertiesLocator Found 6 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestLazyCores_52E42445FD98FFF3-001/tempDir-001
[junit4] 2> 1857313 INFO (TEST-TestLazyCores.testBadConfigsGenerateErrors-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [badSchema1, core1, badConfig2, badConfig1, badSchema2, core2]
[junit4] 2> 1857313 INFO (TEST-TestLazyCores.testBadConfigsGenerateErrors-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1857320 INFO (coreLoadExecutor-5652-thread-1) [ x:badSchema1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 1857320 INFO (coreLoadExecutor-5652-thread-2) [ x:core1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 1857325 INFO (coreLoadExecutor-5652-thread-2) [ x:core1] o.a.s.s.IndexSchema [core1] Schema name=tiny
[junit4] 2> 1857329 INFO (coreLoadExecutor-5652-thread-3) [ x:badConfig2] o.a.s.c.Config Ignoring exception matching \QSAXParseException\E
[junit4] 2> 1857329 INFO (coreLoadExecutor-5652-thread-1) [ x:badSchema1] o.a.s.c.Config Ignoring exception matching \QSAXParseException\E
[junit4] 2> 1857330 INFO (coreLoadExecutor-5652-thread-2) [ x:core1] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 1857330 INFO (coreLoadExecutor-5652-thread-2) [ x:core1] o.a.s.c.CoreContainer Creating SolrCore 'core1' using configuration from instancedir /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestLazyCores_52E42445FD98FFF3-001/tempDir-001/core1, trusted=true
[junit4] 2> 1857330 INFO (coreLoadExecutor-5652-thread-2) [ x:core1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.core1' (registry 'solr.core.core1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4473d4be
[junit4] 2> 1857331 INFO (coreLoadExecutor-5652-thread-2) [ x:core1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1857331 INFO (coreLoadExecutor-5652-thread-2) [ x:core1] o.a.s.c.SolrCore [[core1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestLazyCores_52E42445FD98FFF3-001/tempDir-001/core1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestLazyCores_52E42445FD98FFF3-001/tempDir-001/core1/data/]
[junit4] 2> 1857331 INFO (coreLoadExecutor-5652-thread-1) [ x:badConfig1] o.a.s.c.Config Ignoring exception matching \QSAXParseException\E
[junit4] 2> 1857333 INFO (coreLoadExecutor-5652-thread-3) [ x:badSchema2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 1857333 INFO (coreLoadExecutor-5652-thread-2) [ x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=17, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1857349 INFO (coreLoadExecutor-5652-thread-3) [ x:badSchema2] o.a.s.c.Config Ignoring exception matching \QSAXParseException\E
[junit4] 2> 1857349 INFO (coreLoadExecutor-5652-thread-1) [ x:core2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 1857353 INFO (coreLoadExecutor-5652-thread-1) [ x:core2] o.a.s.s.IndexSchema [core2] Schema name=tiny
[junit4] 2> 1857354 INFO (coreLoadExecutor-5652-thread-1) [ x:core2] o.a.s.s.IndexSchema Loaded schema tiny/1.1 with uniqueid field id
[junit4] 2> 1857354 INFO (coreLoadExecutor-5652-thread-1) [ x:core2] o.a.s.c.CoreContainer Creating SolrCore 'core2' using configuration from instancedir /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestLazyCores_52E42445FD98FFF3-001/tempDir-001/core2, trusted=true
[junit4] 2> 1857354 INFO (coreLoadExecutor-5652-thread-1) [ x:core2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.core2' (registry 'solr.core.core2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4473d4be
[junit4] 2> 1857354 INFO (coreLoadExecutor-5652-thread-1) [ x:core2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1857354 INFO (coreLoadExecutor-5652-thread-1) [ x:core2] o.a.s.c.SolrCore [[core2] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestLazyCores_52E42445FD98FFF3-001/tempDir-001/core2], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestLazyCores_52E42445FD98FFF3-001/tempDir-001/core2/data/]
[junit4] 2> 1857355 INFO (coreLoadExecutor-5652-thread-1) [ x:core2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=17, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1857396 INFO (coreLoadExecutor-5652-thread-2) [ x:core1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1857396 INFO (coreLoadExecutor-5652-thread-2) [ x:core1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1857397 INFO (coreLoadExecutor-5652-thread-2) [ x:core1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=46, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1039003226910042]
[junit4] 2> 1857397 INFO (coreLoadExecutor-5652-thread-2) [ x:core1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4303b4fb[core1] main]
[junit4] 2> 1857397 INFO (coreLoadExecutor-5652-thread-2) [ x:core1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestLazyCores_52E42445FD98FFF3-001/tempDir-001/core1/conf
[junit4] 2> 1857398 INFO (coreLoadExecutor-5652-thread-2) [ x:core1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1857398 INFO (searcherExecutor-5653-thread-1-processing-x:core1) [ x:core1] o.a.s.c.SolrCore [core1] Registered new searcher Searcher@4303b4fb[core1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1857410 INFO (coreLoadExecutor-5652-thread-1) [ x:core2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1857410 INFO (coreLoadExecutor-5652-thread-1) [ x:core2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1857411 INFO (coreLoadExecutor-5652-thread-1) [ x:core2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=46, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1039003226910042]
[junit4] 2> 1857411 INFO (coreLoadExecutor-5652-thread-1) [ x:core2] o.a.s.s.SolrIndexSearcher Opening [Searcher@26def5e5[core2] main]
[junit4] 2> 1857412 INFO (coreLoadExecutor-5652-thread-1) [ x:core2] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestLazyCores_52E42445FD98FFF3-001/tempDir-001/core2/conf
[junit4] 2> 1857412 INFO (coreLoadExecutor-5652-thread-1) [ x:core2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1857414 INFO (TEST-TestLazyCores.testBadConfigsGenerateErrors-seed#[52E42445FD98FFF3]) [ ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1857414 INFO (TEST-TestLazyCores.testBadConfigsGenerateErrors-seed#[52E42445FD98FFF3]) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@6001efd1 commitCommandVersion:0
[junit4] 2> 1857414 INFO (searcherExecutor-5654-thread-1-processing-x:core2) [ x:core2] o.a.s.c.SolrCore [core2] Registered new searcher Searcher@26def5e5[core2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1857449 INFO (TEST-TestLazyCores.testBadConfigsGenerateErrors-seed#[52E42445FD98FFF3]) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@1a35bc15[core1] main]
[junit4] 2> 1857449 INFO (TEST-TestLazyCores.testBadConfigsGenerateErrors-seed#[52E42445FD98FFF3]) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1857450 INFO (searcherExecutor-5653-thread-1) [ ] o.a.s.c.SolrCore [core1] Registered new searcher Searcher@1a35bc15[core1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.3.0):c7)))}
[junit4] 2> 1857451 INFO (TEST-TestLazyCores.testBadConfigsGenerateErrors-seed#[52E42445FD98FFF3]) [ ] o.a.s.c.S.Request [core1] webapp=null path=null params={q={!prefix+f%3Dv_t}hel&wt=xml} hits=2 status=0 QTime=0
[junit4] 2> 1857453 INFO (TEST-TestLazyCores.testBadConfigsGenerateErrors-seed#[52E42
[...truncated too long message...]
oolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([52E42445FD98FFF3]:0)
[junit4] Completed [407/751 (2!)] on J1 in 30.03s, 9 tests, 3 errors <<< FAILURES!
[...truncated 35661 lines...]
-ecj-javadoc-lint-src:
[mkdir] Created dir: /tmp/ecj403393266
[ecj-lint] Compiling 188 source files to /tmp/ecj403393266
[ecj-lint] ----------
[ecj-lint] 1. WARNING in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/test-framework/src/java/org/apache/lucene/analysis/BaseTokenStreamTestCase.java (at line 801)
[ecj-lint] ts = a.tokenStream("dummy", useCharFilter ? new MockCharFilter(reader, remainder) : reader);
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'ts' is not closed at this location
[ecj-lint] ----------
[ecj-lint] 2. WARNING in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/test-framework/src/java/org/apache/lucene/analysis/BaseTokenStreamTestCase.java (at line 837)
[ecj-lint] reader = new MockReaderWrapper(random, reader);
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'reader' is not closed at this location
[ecj-lint] ----------
[ecj-lint] 3. WARNING in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/test-framework/src/java/org/apache/lucene/analysis/BaseTokenStreamTestCase.java (at line 913)
[ecj-lint] reader = new MockReaderWrapper(random, reader);
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'reader' is not closed at this location
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 4. WARNING in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/test-framework/src/java/org/apache/lucene/codecs/mockrandom/MockRandomPostingsFormat.java (at line 260)
[ecj-lint] throw new AssertionError();
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'postingsWriter' is not closed at this location
[ecj-lint] ----------
[ecj-lint] 5. WARNING in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/test-framework/src/java/org/apache/lucene/codecs/mockrandom/MockRandomPostingsFormat.java (at line 393)
[ecj-lint] throw new AssertionError();
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'postingsReader' is not closed at this location
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 6. WARNING in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/test-framework/src/java/org/apache/lucene/geo/BaseGeoPointTestCase.java (at line 1271)
[ecj-lint] RandomIndexWriter writer = new RandomIndexWriter(random(), dir, iwc);
[ecj-lint] ^^^^^^
[ecj-lint] Resource leak: 'writer' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 7. WARNING in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/test-framework/src/java/org/apache/lucene/index/BaseCompressingDocValuesFormatTestCase.java (at line 47)
[ecj-lint] final IndexWriter iwriter = new IndexWriter(dir, iwc);
[ecj-lint] ^^^^^^^
[ecj-lint] Resource leak: 'iwriter' is never closed
[ecj-lint] ----------
[ecj-lint] 8. WARNING in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/test-framework/src/java/org/apache/lucene/index/BaseCompressingDocValuesFormatTestCase.java (at line 81)
[ecj-lint] final IndexWriter iwriter = new IndexWriter(dir, iwc);
[ecj-lint] ^^^^^^^
[ecj-lint] Resource leak: 'iwriter' is never closed
[ecj-lint] ----------
[ecj-lint] 9. WARNING in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/test-framework/src/java/org/apache/lucene/index/BaseCompressingDocValuesFormatTestCase.java (at line 108)
[ecj-lint] final IndexWriter iwriter = new IndexWriter(dir, iwc);
[ecj-lint] ^^^^^^^
[ecj-lint] Resource leak: 'iwriter' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 10. WARNING in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/test-framework/src/java/org/apache/lucene/index/BasePointsFormatTestCase.java (at line 622)
[ecj-lint] w = new RandomIndexWriter(random(), dir, iwc);
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'w' is not closed at this location
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 11. WARNING in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/test-framework/src/java/org/apache/lucene/index/BasePostingsFormatTestCase.java (at line 314)
[ecj-lint] Analyzer analyzer = new MockAnalyzer(random());
[ecj-lint] ^^^^^^^^
[ecj-lint] Resource leak: 'analyzer' is never closed
[ecj-lint] ----------
[ecj-lint] 12. WARNING in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/test-framework/src/java/org/apache/lucene/index/BasePostingsFormatTestCase.java (at line 539)
[ecj-lint] LineFileDocs docs = new LineFileDocs(random());
[ecj-lint] ^^^^
[ecj-lint] Resource leak: 'docs' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 13. WARNING in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/test-framework/src/java/org/apache/lucene/index/BaseStoredFieldsFormatTestCase.java (at line 464)
[ecj-lint] throw ex.get();
[ecj-lint] ^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'iw' is not closed at this location
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 14. WARNING in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/test-framework/src/java/org/apache/lucene/search/ShardSearchingTestBase.java (at line 555)
[ecj-lint] final LineFileDocs docs = new LineFileDocs(random());
[ecj-lint] ^^^^
[ecj-lint] Resource leak: 'docs' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 15. WARNING in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/test-framework/src/java/org/apache/lucene/store/MockDirectoryWrapper.java (at line 881)
[ecj-lint] new IndexWriter(in, iwc).rollback();
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 16. ERROR in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/test-framework/src/java/org/apache/lucene/util/LuceneTestCase.java (at line 144)
[ecj-lint] import static com.carrotsearch.randomizedtesting.RandomizedTest.frequently;
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] The import com.carrotsearch.randomizedtesting.RandomizedTest.frequently is never used
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 17. WARNING in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/test-framework/src/java/org/apache/lucene/util/TestUtil.java (at line 1310)
[ecj-lint] fs = ffs.getDelegate();
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'fs' is not closed at this location
[ecj-lint] ----------
[ecj-lint] 18. WARNING in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/test-framework/src/java/org/apache/lucene/util/TestUtil.java (at line 1324)
[ecj-lint] fs = ffs.getDelegate();
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'fs' is not closed at this location
[ecj-lint] ----------
[ecj-lint] 19. WARNING in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/test-framework/src/java/org/apache/lucene/util/TestUtil.java (at line 1324)
[ecj-lint] fs = ffs.getDelegate();
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'fs' is not closed at this location
[ecj-lint] ----------
[ecj-lint] 20. WARNING in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/test-framework/src/java/org/apache/lucene/util/TestUtil.java (at line 1346)
[ecj-lint] fs = ffs.getDelegate();
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'fs' is not closed at this location
[ecj-lint] ----------
[ecj-lint] 21. WARNING in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/test-framework/src/java/org/apache/lucene/util/TestUtil.java (at line 1346)
[ecj-lint] fs = ffs.getDelegate();
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'fs' is not closed at this location
[ecj-lint] ----------
[ecj-lint] 22. WARNING in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/test-framework/src/java/org/apache/lucene/util/TestUtil.java (at line 1366)
[ecj-lint] fs = ffs.getDelegate();
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'fs' is not closed at this location
[ecj-lint] ----------
[ecj-lint] 23. WARNING in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/test-framework/src/java/org/apache/lucene/util/TestUtil.java (at line 1385)
[ecj-lint] fs = ffs.getDelegate();
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'fs' is not closed at this location
[ecj-lint] ----------
[ecj-lint] 23 problems (1 error, 22 warnings)
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/build.xml:835: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/build.xml:101: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/build.xml:203: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/common-build.xml:2105: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/common-build.xml:2144: Compile failed; see the compiler error output for details.
Total time: 61 minutes 22 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-7.x-Linux (64bit/jdk1.8.0_144) - Build # 960
- Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/960/
Java: 64bit/jdk1.8.0_144 -XX:+UseCompressedOops -XX:+UseSerialGC
1 tests failed.
FAILED: org.apache.solr.cloud.TestDeleteCollectionOnDownNodes.deleteCollectionWithDownNodes
Error Message:
Error from server at http://127.0.0.1:33431/solr: create the collection time out:180s
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:33431/solr: create the collection time out:180s
at __randomizedtesting.SeedInfo.seed([14BB2B640D0472EB:8A8E4F9C2B273E63]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1103)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:883)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.TestDeleteCollectionOnDownNodes.deleteCollectionWithDownNodes(TestDeleteCollectionOnDownNodes.java:40)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12321 lines...]
[junit4] Suite: org.apache.solr.cloud.TestDeleteCollectionOnDownNodes
[junit4] 2> 710071 INFO (SUITE-TestDeleteCollectionOnDownNodes-seed#[14BB2B640D0472EB]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/init-core-data-001
[junit4] 2> 710072 WARN (SUITE-TestDeleteCollectionOnDownNodes-seed#[14BB2B640D0472EB]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
[junit4] 2> 710072 INFO (SUITE-TestDeleteCollectionOnDownNodes-seed#[14BB2B640D0472EB]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 710074 INFO (SUITE-TestDeleteCollectionOnDownNodes-seed#[14BB2B640D0472EB]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 710074 INFO (SUITE-TestDeleteCollectionOnDownNodes-seed#[14BB2B640D0472EB]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001
[junit4] 2> 710074 INFO (SUITE-TestDeleteCollectionOnDownNodes-seed#[14BB2B640D0472EB]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 710076 INFO (Thread-680) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 710076 INFO (Thread-680) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 710084 ERROR (Thread-680) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 710176 INFO (SUITE-TestDeleteCollectionOnDownNodes-seed#[14BB2B640D0472EB]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:41153
[junit4] 2> 710178 INFO (zkConnectionManagerCallback-798-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 710180 INFO (jetty-launcher-795-thread-3) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 710180 INFO (jetty-launcher-795-thread-1) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 710180 INFO (jetty-launcher-795-thread-2) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 710180 INFO (jetty-launcher-795-thread-4) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 710182 INFO (jetty-launcher-795-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@62972781{/solr,null,AVAILABLE}
[junit4] 2> 710182 INFO (jetty-launcher-795-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3821a6b0{/solr,null,AVAILABLE}
[junit4] 2> 710182 INFO (jetty-launcher-795-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@487eaa22{/solr,null,AVAILABLE}
[junit4] 2> 710183 INFO (jetty-launcher-795-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2ee958ab{HTTP/1.1,[http/1.1]}{127.0.0.1:35843}
[junit4] 2> 710183 INFO (jetty-launcher-795-thread-1) [ ] o.e.j.s.Server Started @711309ms
[junit4] 2> 710183 INFO (jetty-launcher-795-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35843}
[junit4] 2> 710183 ERROR (jetty-launcher-795-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 710183 INFO (jetty-launcher-795-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 710183 INFO (jetty-launcher-795-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 710183 INFO (jetty-launcher-795-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 710183 INFO (jetty-launcher-795-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-08T19:20:11.025Z
[junit4] 2> 710184 INFO (jetty-launcher-795-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@992c446{HTTP/1.1,[http/1.1]}{127.0.0.1:33849}
[junit4] 2> 710184 INFO (jetty-launcher-795-thread-3) [ ] o.e.j.s.Server Started @711310ms
[junit4] 2> 710184 INFO (jetty-launcher-795-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4f564c49{/solr,null,AVAILABLE}
[junit4] 2> 710184 INFO (jetty-launcher-795-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33849}
[junit4] 2> 710184 ERROR (jetty-launcher-795-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 710184 INFO (jetty-launcher-795-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 710184 INFO (jetty-launcher-795-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 710184 INFO (jetty-launcher-795-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 710184 INFO (jetty-launcher-795-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-08T19:20:11.026Z
[junit4] 2> 710184 INFO (jetty-launcher-795-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@d04d350{HTTP/1.1,[http/1.1]}{127.0.0.1:35809}
[junit4] 2> 710185 INFO (jetty-launcher-795-thread-4) [ ] o.e.j.s.Server Started @711311ms
[junit4] 2> 710185 INFO (jetty-launcher-795-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35809}
[junit4] 2> 710185 ERROR (jetty-launcher-795-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 710185 INFO (jetty-launcher-795-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 710185 INFO (jetty-launcher-795-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 710185 INFO (jetty-launcher-795-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 710185 INFO (jetty-launcher-795-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-08T19:20:11.027Z
[junit4] 2> 710186 INFO (zkConnectionManagerCallback-800-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 710186 INFO (jetty-launcher-795-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 710190 INFO (zkConnectionManagerCallback-802-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 710190 INFO (jetty-launcher-795-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@370e5a0f{HTTP/1.1,[http/1.1]}{127.0.0.1:33431}
[junit4] 2> 710190 INFO (jetty-launcher-795-thread-2) [ ] o.e.j.s.Server Started @711316ms
[junit4] 2> 710190 INFO (jetty-launcher-795-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33431}
[junit4] 2> 710190 ERROR (jetty-launcher-795-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 710190 INFO (jetty-launcher-795-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 710190 INFO (jetty-launcher-795-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 710190 INFO (jetty-launcher-795-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 710190 INFO (jetty-launcher-795-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-08T19:20:11.032Z
[junit4] 2> 710195 INFO (jetty-launcher-795-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 710206 INFO (zkConnectionManagerCallback-804-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 710206 INFO (jetty-launcher-795-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 710216 INFO (jetty-launcher-795-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41153/solr
[junit4] 2> 710216 INFO (zkConnectionManagerCallback-806-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 710218 INFO (jetty-launcher-795-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41153/solr
[junit4] 2> 710218 INFO (jetty-launcher-795-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 710226 INFO (zkConnectionManagerCallback-811-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 710229 INFO (zkConnectionManagerCallback-814-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 710234 INFO (zkConnectionManagerCallback-816-thread-1-processing-n:127.0.0.1:35843_solr) [n:127.0.0.1:35843_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 710234 INFO (jetty-launcher-795-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41153/solr
[junit4] 2> 710238 INFO (zkConnectionManagerCallback-819-thread-1-processing-n:127.0.0.1:33849_solr) [n:127.0.0.1:33849_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 710238 INFO (jetty-launcher-795-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41153/solr
[junit4] 2> 710246 INFO (zkConnectionManagerCallback-823-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 710246 INFO (zkConnectionManagerCallback-827-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 710256 INFO (jetty-launcher-795-thread-3) [n:127.0.0.1:33849_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 710256 INFO (jetty-launcher-795-thread-3) [n:127.0.0.1:33849_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33849_solr
[junit4] 2> 710257 INFO (jetty-launcher-795-thread-3) [n:127.0.0.1:33849_solr ] o.a.s.c.Overseer Overseer (id=99140292504387592-127.0.0.1:33849_solr-n_0000000000) starting
[junit4] 2> 710258 INFO (zkConnectionManagerCallback-830-thread-1-processing-n:127.0.0.1:35809_solr) [n:127.0.0.1:35809_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 710261 INFO (jetty-launcher-795-thread-4) [n:127.0.0.1:35809_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 710262 INFO (jetty-launcher-795-thread-4) [n:127.0.0.1:35809_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35809_solr
[junit4] 2> 710270 INFO (jetty-launcher-795-thread-1) [n:127.0.0.1:35843_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 710274 INFO (zkCallback-829-thread-1-processing-n:127.0.0.1:35809_solr) [n:127.0.0.1:35809_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 710274 INFO (zkConnectionManagerCallback-832-thread-1-processing-n:127.0.0.1:33431_solr) [n:127.0.0.1:33431_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 710275 INFO (jetty-launcher-795-thread-1) [n:127.0.0.1:35843_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 710276 INFO (jetty-launcher-795-thread-1) [n:127.0.0.1:35843_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35843_solr
[junit4] 2> 710277 INFO (jetty-launcher-795-thread-2) [n:127.0.0.1:33431_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 710280 INFO (jetty-launcher-795-thread-2) [n:127.0.0.1:33431_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 710280 INFO (jetty-launcher-795-thread-2) [n:127.0.0.1:33431_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33431_solr
[junit4] 2> 710282 INFO (zkCallback-829-thread-1-processing-n:127.0.0.1:35809_solr) [n:127.0.0.1:35809_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 710286 INFO (zkCallback-818-thread-1-processing-n:127.0.0.1:33849_solr) [n:127.0.0.1:33849_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 710294 INFO (zkCallback-815-thread-1-processing-n:127.0.0.1:35843_solr) [n:127.0.0.1:35843_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 710294 INFO (zkCallback-818-thread-2-processing-n:127.0.0.1:33849_solr) [n:127.0.0.1:33849_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 710310 INFO (zkCallback-831-thread-1-processing-n:127.0.0.1:33431_solr) [n:127.0.0.1:33431_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 710313 INFO (jetty-launcher-795-thread-3) [n:127.0.0.1:33849_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33849_solr
[junit4] 2> 710314 INFO (zkCallback-829-thread-1-processing-n:127.0.0.1:35809_solr) [n:127.0.0.1:35809_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 710314 INFO (zkCallback-815-thread-1-processing-n:127.0.0.1:35843_solr) [n:127.0.0.1:35843_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 710314 INFO (zkCallback-818-thread-2-processing-n:127.0.0.1:33849_solr) [n:127.0.0.1:33849_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 710315 INFO (zkCallback-831-thread-1-processing-n:127.0.0.1:33431_solr) [n:127.0.0.1:33431_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 710367 INFO (jetty-launcher-795-thread-1) [n:127.0.0.1:35843_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35843.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@63475939
[junit4] 2> 710397 INFO (jetty-launcher-795-thread-1) [n:127.0.0.1:35843_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35843.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@63475939
[junit4] 2> 710397 INFO (jetty-launcher-795-thread-1) [n:127.0.0.1:35843_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35843.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@63475939
[junit4] 2> 710398 INFO (jetty-launcher-795-thread-1) [n:127.0.0.1:35843_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node1/.
[junit4] 2> 710398 INFO (jetty-launcher-795-thread-4) [n:127.0.0.1:35809_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35809.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@63475939
[junit4] 2> 710402 INFO (jetty-launcher-795-thread-3) [n:127.0.0.1:33849_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33849.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@63475939
[junit4] 2> 710406 INFO (jetty-launcher-795-thread-4) [n:127.0.0.1:35809_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35809.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@63475939
[junit4] 2> 710406 INFO (jetty-launcher-795-thread-4) [n:127.0.0.1:35809_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35809.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@63475939
[junit4] 2> 710407 INFO (jetty-launcher-795-thread-4) [n:127.0.0.1:35809_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node4/.
[junit4] 2> 710410 INFO (jetty-launcher-795-thread-3) [n:127.0.0.1:33849_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33849.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@63475939
[junit4] 2> 710411 INFO (jetty-launcher-795-thread-3) [n:127.0.0.1:33849_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33849.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@63475939
[junit4] 2> 710412 INFO (jetty-launcher-795-thread-3) [n:127.0.0.1:33849_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node3/.
[junit4] 2> 710413 INFO (jetty-launcher-795-thread-2) [n:127.0.0.1:33431_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33431.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@63475939
[junit4] 2> 710417 INFO (jetty-launcher-795-thread-2) [n:127.0.0.1:33431_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33431.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@63475939
[junit4] 2> 710418 INFO (jetty-launcher-795-thread-2) [n:127.0.0.1:33431_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33431.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@63475939
[junit4] 2> 710419 INFO (jetty-launcher-795-thread-2) [n:127.0.0.1:33431_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node2/.
[junit4] 2> 710526 INFO (zkConnectionManagerCallback-837-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 710534 INFO (zkConnectionManagerCallback-841-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 710534 INFO (SUITE-TestDeleteCollectionOnDownNodes-seed#[14BB2B640D0472EB]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 710535 INFO (SUITE-TestDeleteCollectionOnDownNodes-seed#[14BB2B640D0472EB]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41153/solr ready
[junit4] 2> 710554 INFO (TEST-TestDeleteCollectionOnDownNodes.deleteCollectionWithDownNodes-seed#[14BB2B640D0472EB]) [ ] o.a.s.SolrTestCaseJ4 ###Starting deleteCollectionWithDownNodes
[junit4] 2> 710557 INFO (qtp247979140-4027) [n:127.0.0.1:33431_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=3&collection.configName=conf&maxShardsPerNode=3&name=halfdeletedcollection2&nrtReplicas=3&action=CREATE&numShards=4&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 710563 INFO (OverseerThreadFactory-1426-thread-1-processing-n:127.0.0.1:33849_solr) [n:127.0.0.1:33849_solr ] o.a.s.c.CreateCollectionCmd Create collection halfdeletedcollection2
[junit4] 2> 710667 INFO (OverseerStateUpdate-99140292504387592-127.0.0.1:33849_solr-n_0000000000) [n:127.0.0.1:33849_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"halfdeletedcollection2",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"halfdeletedcollection2_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:35809/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 710668 INFO (OverseerStateUpdate-99140292504387592-127.0.0.1:33849_solr-n_0000000000) [n:127.0.0.1:33849_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"halfdeletedcollection2",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"halfdeletedcollection2_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:35843/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 710670 INFO (OverseerStateUpdate-99140292504387592-127.0.0.1:33849_solr-n_0000000000) [n:127.0.0.1:33849_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"halfdeletedcollection2",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"halfdeletedcollection2_shard1_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:33849/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 710671 INFO (OverseerStateUpdate-99140292504387592-127.0.0.1:33849_solr-n_0000000000) [n:127.0.0.1:33849_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"halfdeletedcollection2",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"halfdeletedcollection2_shard2_replica_n6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:33431/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 710672 INFO (OverseerStateUpdate-99140292504387592-127.0.0.1:33849_solr-n_0000000000) [n:127.0.0.1:33849_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"halfdeletedcollection2",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"halfdeletedcollection2_shard2_replica_n8",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:35809/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 710673 INFO (OverseerStateUpdate-99140292504387592-127.0.0.1:33849_solr-n_0000000000) [n:127.0.0.1:33849_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"halfdeletedcollection2",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"halfdeletedcollection2_shard2_replica_n10",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:35843/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 710674 INFO (OverseerStateUpdate-99140292504387592-127.0.0.1:33849_solr-n_0000000000) [n:127.0.0.1:33849_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"halfdeletedcollection2",
[junit4] 2> "shard":"shard3",
[junit4] 2> "core":"halfdeletedcollection2_shard3_replica_n12",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:33849/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 710674 INFO (OverseerStateUpdate-99140292504387592-127.0.0.1:33849_solr-n_0000000000) [n:127.0.0.1:33849_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"halfdeletedcollection2",
[junit4] 2> "shard":"shard3",
[junit4] 2> "core":"halfdeletedcollection2_shard3_replica_n14",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:33431/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 710675 INFO (OverseerStateUpdate-99140292504387592-127.0.0.1:33849_solr-n_0000000000) [n:127.0.0.1:33849_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"halfdeletedcollection2",
[junit4] 2> "shard":"shard3",
[junit4] 2> "core":"halfdeletedcollection2_shard3_replica_n16",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:35809/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 710676 INFO (OverseerStateUpdate-99140292504387592-127.0.0.1:33849_solr-n_0000000000) [n:127.0.0.1:33849_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"halfdeletedcollection2",
[junit4] 2> "shard":"shard4",
[junit4] 2> "core":"halfdeletedcollection2_shard4_replica_n18",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:35843/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 710676 INFO (OverseerStateUpdate-99140292504387592-127.0.0.1:33849_solr-n_0000000000) [n:127.0.0.1:33849_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"halfdeletedcollection2",
[junit4] 2> "shard":"shard4",
[junit4] 2> "core":"halfdeletedcollection2_shard4_replica_n20",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:33849/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 710677 INFO (OverseerStateUpdate-99140292504387592-127.0.0.1:33849_solr-n_0000000000) [n:127.0.0.1:33849_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"halfdeletedcollection2",
[junit4] 2> "shard":"shard4",
[junit4] 2> "core":"halfdeletedcollection2_shard4_replica_n22",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:33431/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 710882 INFO (qtp247979140-4017) [n:127.0.0.1:33431_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node17&collection.configName=conf&newCollection=true&name=halfdeletedcollection2_shard3_replica_n14&action=CREATE&numShards=4&collection=halfdeletedcollection2&shard=shard3&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 710882 INFO (qtp247979140-4017) [n:127.0.0.1:33431_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 710883 INFO (qtp2025042261-4012) [n:127.0.0.1:35843_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node21&collection.configName=conf&newCollection=true&name=halfdeletedcollection2_shard4_replica_n18&action=CREATE&numShards=4&collection=halfdeletedcollection2&shard=shard4&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 710883 INFO (qtp2025042261-4012) [n:127.0.0.1:35843_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 710886 INFO (qtp2025042261-4011) [n:127.0.0.1:35843_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node13&collection.configName=conf&newCollection=true&name=halfdeletedcollection2_shard2_replica_n10&action=CREATE&numShards=4&collection=halfdeletedcollection2&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 710886 INFO (qtp2025042261-4008) [n:127.0.0.1:35843_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf&newCollection=true&name=halfdeletedcollection2_shard1_replica_n2&action=CREATE&numShards=4&collection=halfdeletedcollection2&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 710887 INFO (qtp1557452267-4039) [n:127.0.0.1:35809_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=halfdeletedcollection2_shard1_replica_n1&action=CREATE&numShards=4&collection=halfdeletedcollection2&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 710887 INFO (qtp247979140-4034) [n:127.0.0.1:33431_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node9&collection.configName=conf&newCollection=true&name=halfdeletedcollection2_shard2_replica_n6&action=CREATE&numShards=4&collection=halfdeletedcollection2&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 710887 INFO (qtp1557452267-4039) [n:127.0.0.1:35809_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 710898 INFO (qtp1557452267-4041) [n:127.0.0.1:35809_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node11&collection.configName=conf&newCollection=true&name=halfdeletedcollection2_shard2_replica_n8&action=CREATE&numShards=4&collection=halfdeletedcollection2&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 710899 INFO (qtp903444575-4026) [n:127.0.0.1:33849_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf&newCollection=true&name=halfdeletedcollection2_shard1_replica_n4&action=CREATE&numShards=4&collection=halfdeletedcollection2&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 710899 INFO (qtp903444575-4026) [n:127.0.0.1:33849_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 710902 INFO (qtp247979140-4019) [n:127.0.0.1:33431_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node24&collection.configName=conf&newCollection=true&name=halfdeletedcollection2_shard4_replica_n22&action=CREATE&numShards=4&collection=halfdeletedcollection2&shard=shard4&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 710907 INFO (qtp1557452267-4028) [n:127.0.0.1:35809_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node19&collection.configName=conf&newCollection=true&name=halfdeletedcollection2_shard3_replica_n16&action=CREATE&numShards=4&collection=halfdeletedcollection2&shard=shard3&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 710909 INFO (qtp903444575-4029) [n:127.0.0.1:33849_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node15&collection.configName=conf&newCollection=true&name=halfdeletedcollection2_shard3_replica_n12&action=CREATE&numShards=4&collection=halfdeletedcollection2&shard=shard3&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 710909 INFO (qtp903444575-4007) [n:127.0.0.1:33849_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node23&collection.configName=conf&newCollection=true&name=halfdeletedcollection2_shard4_replica_n20&action=CREATE&numShards=4&collection=halfdeletedcollection2&shard=shard4&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 711011 INFO (zkCallback-818-thread-2-processing-n:127.0.0.1:33849_solr) [n:127.0.0.1:33849_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfdeletedcollection2/state.json] for collection [halfdeletedcollection2] has occurred - updating... (live nodes size: [4])
[junit4] 2> 711011 INFO (zkCallback-815-thread-1-processing-n:127.0.0.1:35843_solr) [n:127.0.0.1:35843_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfdeletedcollection2/state.json] for collection [halfdeletedcollection2] has occurred - updating... (live nodes size: [4])
[junit4] 2> 711011 INFO (zkCallback-831-thread-1-processing-n:127.0.0.1:33431_solr) [n:127.0.0.1:33431_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfdeletedcollection2/state.json] for collection [halfdeletedcollection2] has occurred - updating... (live nodes size: [4])
[junit4] 2> 711011 INFO (zkCallback-831-thread-2-processing-n:127.0.0.1:33431_solr) [n:127.0.0.1:33431_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfdeletedcollection2/state.json] for collection [halfdeletedcollection2] has occurred - updating... (live nodes size: [4])
[junit4] 2> 711011 INFO (zkCallback-815-thread-2-processing-n:127.0.0.1:35843_solr) [n:127.0.0.1:35843_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfdeletedcollection2/state.json] for collection [halfdeletedcollection2] has occurred - updating... (live nodes size: [4])
[junit4] 2> 711011 INFO (zkCallback-818-thread-1-processing-n:127.0.0.1:33849_solr) [n:127.0.0.1:33849_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfdeletedcollection2/state.json] for collection [halfdeletedcollection2] has occurred - updating... (live nodes size: [4])
[junit4] 2> 711011 INFO (zkCallback-829-thread-1-processing-n:127.0.0.1:35809_solr) [n:127.0.0.1:35809_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfdeletedcollection2/state.json] for collection [halfdeletedcollection2] has occurred - updating... (live nodes size: [4])
[junit4] 2> 711011 INFO (zkCallback-815-thread-3-processing-n:127.0.0.1:35843_solr) [n:127.0.0.1:35843_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfdeletedcollection2/state.json] for collection [halfdeletedcollection2] has occurred - updating... (live nodes size: [4])
[junit4] 2> 711915 INFO (qtp1557452267-4041) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard2 r:core_node11 x:halfdeletedcollection2_shard2_replica_n8] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 711916 INFO (qtp2025042261-4012) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard4 r:core_node21 x:halfdeletedcollection2_shard4_replica_n18] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 711936 INFO (qtp1557452267-4039) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard1 r:core_node3 x:halfdeletedcollection2_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 711939 INFO (qtp2025042261-4008) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard1 r:core_node5 x:halfdeletedcollection2_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 711939 INFO (qtp247979140-4017) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard3 r:core_node17 x:halfdeletedcollection2_shard3_replica_n14] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 711942 INFO (qtp903444575-4026) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard1 r:core_node7 x:halfdeletedcollection2_shard1_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 711943 INFO (qtp247979140-4034) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard2 r:core_node9 x:halfdeletedcollection2_shard2_replica_n6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 711946 INFO (qtp247979140-4019) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard4 r:core_node24 x:halfdeletedcollection2_shard4_replica_n22] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 711959 INFO (qtp1557452267-4028) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard3 r:core_node19 x:halfdeletedcollection2_shard3_replica_n16] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 711964 INFO (qtp247979140-4034) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard2 r:core_node9 x:halfdeletedcollection2_shard2_replica_n6] o.a.s.s.IndexSchema [halfdeletedcollection2_shard2_replica_n6] Schema name=minimal
[junit4] 2> 711964 INFO (qtp903444575-4026) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard1 r:core_node7 x:halfdeletedcollection2_shard1_replica_n4] o.a.s.s.IndexSchema [halfdeletedcollection2_shard1_replica_n4] Schema name=minimal
[junit4] 2> 711964 INFO (qtp2025042261-4008) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard1 r:core_node5 x:halfdeletedcollection2_shard1_replica_n2] o.a.s.s.IndexSchema [halfdeletedcollection2_shard1_replica_n2] Schema name=minimal
[junit4] 2> 711965 INFO (qtp903444575-4029) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard3 r:core_node15 x:halfdeletedcollection2_shard3_replica_n12] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 711967 INFO (qtp247979140-4017) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard3 r:core_node17 x:halfdeletedcollection2_shard3_replica_n14] o.a.s.s.IndexSchema [halfdeletedcollection2_shard3_replica_n14] Schema name=minimal
[junit4] 2> 711967 INFO (qtp1557452267-4039) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard1 r:core_node3 x:halfdeletedcollection2_shard1_replica_n1] o.a.s.s.IndexSchema [halfdeletedcollection2_shard1_replica_n1] Schema name=minimal
[junit4] 2> 711968 INFO (qtp1557452267-4041) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard2 r:core_node11 x:halfdeletedcollection2_shard2_replica_n8] o.a.s.s.IndexSchema [halfdeletedcollection2_shard2_replica_n8] Schema name=minimal
[junit4] 2> 711968 INFO (qtp903444575-4007) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard4 r:core_node23 x:halfdeletedcollection2_shard4_replica_n20] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 711970 INFO (qtp247979140-4034) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard2 r:core_node9 x:halfdeletedcollection2_shard2_replica_n6] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 711970 INFO (qtp247979140-4034) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard2 r:core_node9 x:halfdeletedcollection2_shard2_replica_n6] o.a.s.c.CoreContainer Creating SolrCore 'halfdeletedcollection2_shard2_replica_n6' using configuration from collection halfdeletedcollection2, trusted=true
[junit4] 2> 711970 INFO (qtp903444575-4026) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard1 r:core_node7 x:halfdeletedcollection2_shard1_replica_n4] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 711970 INFO (qtp903444575-4026) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard1 r:core_node7 x:halfdeletedcollection2_shard1_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'halfdeletedcollection2_shard1_replica_n4' using configuration from collection halfdeletedcollection2, trusted=true
[junit4] 2> 711970 INFO (qtp2025042261-4008) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard1 r:core_node5 x:halfdeletedcollection2_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 711970 INFO (qtp2025042261-4008) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard1 r:core_node5 x:halfdeletedcollection2_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'halfdeletedcollection2_shard1_replica_n2' using configuration from collection halfdeletedcollection2, trusted=true
[junit4] 2> 711981 INFO (qtp1557452267-4039) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard1 r:core_node3 x:halfdeletedcollection2_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 711981 INFO (qtp1557452267-4039) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard1 r:core_node3 x:halfdeletedcollection2_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'halfdeletedcollection2_shard1_replica_n1' using configuration from collection halfdeletedcollection2, trusted=true
[junit4] 2> 711981 INFO (qtp1557452267-4028) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard3 r:core_node19 x:halfdeletedcollection2_shard3_replica_n16] o.a.s.s.IndexSchema [halfdeletedcollection2_shard3_replica_n16] Schema name=minimal
[junit4] 2> 711981 INFO (qtp903444575-4026) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard1 r:core_node7 x:halfdeletedcollection2_shard1_replica_n4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33849.solr.core.halfdeletedcollection2.shard1.replica_n4' (registry 'solr.core.halfdeletedcollection2.shard1.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@63475939
[junit4] 2> 711981 INFO (qtp247979140-4034) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard2 r:core_node9 x:halfdeletedcollection2_shard2_replica_n6] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33431.solr.core.halfdeletedcollection2.shard2.replica_n6' (registry 'solr.core.halfdeletedcollection2.shard2.replica_n6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@63475939
[junit4] 2> 711981 INFO (qtp247979140-4034) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard2 r:core_node9 x:halfdeletedcollection2_shard2_replica_n6] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 711981 INFO (qtp903444575-4026) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard1 r:core_node7 x:halfdeletedcollection2_shard1_replica_n4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 711981 INFO (qtp903444575-4026) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard1 r:core_node7 x:halfdeletedcollection2_shard1_replica_n4] o.a.s.c.SolrCore [[halfdeletedcollection2_shard1_replica_n4] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node3/halfdeletedcollection2_shard1_replica_n4], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node3/./halfdeletedcollection2_shard1_replica_n4/data/]
[junit4] 2> 711981 INFO (qtp247979140-4034) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard2 r:core_node9 x:halfdeletedcollection2_shard2_replica_n6] o.a.s.c.SolrCore [[halfdeletedcollection2_shard2_replica_n6] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node2/halfdeletedcollection2_shard2_replica_n6], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node2/./halfdeletedcollection2_shard2_replica_n6/data/]
[junit4] 2> 711981 INFO (qtp1557452267-4039) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard1 r:core_node3 x:halfdeletedcollection2_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35809.solr.core.halfdeletedcollection2.shard1.replica_n1' (registry 'solr.core.halfdeletedcollection2.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@63475939
[junit4] 2> 711982 INFO (qtp1557452267-4039) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard1 r:core_node3 x:halfdeletedcollection2_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 711982 INFO (qtp1557452267-4039) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard1 r:core_node3 x:halfdeletedcollection2_shard1_replica_n1] o.a.s.c.SolrCore [[halfdeletedcollection2_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node4/halfdeletedcollection2_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node4/./halfdeletedcollection2_shard1_replica_n1/data/]
[junit4] 2> 711982 INFO (qtp2025042261-4011) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard2 r:core_node13 x:halfdeletedcollection2_shard2_replica_n10] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 711982 INFO (qtp2025042261-4012) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard4 r:core_node21 x:halfdeletedcollection2_shard4_replica_n18] o.a.s.s.IndexSchema [halfdeletedcollection2_shard4_replica_n18] Schema name=minimal
[junit4] 2> 711983 INFO (qtp2025042261-4008) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard1 r:core_node5 x:halfdeletedcollection2_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35843.solr.core.halfdeletedcollection2.shard1.replica_n2' (registry 'solr.core.halfdeletedcollection2.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@63475939
[junit4] 2> 711983 INFO (qtp2025042261-4008) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard1 r:core_node5 x:halfdeletedcollection2_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 711983 INFO (qtp2025042261-4008) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard1 r:core_node5 x:halfdeletedcollection2_shard1_replica_n2] o.a.s.c.SolrCore [[halfdeletedcollection2_shard1_replica_n2] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node1/halfdeletedcollection2_shard1_replica_n2], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node1/./halfdeletedcollection2_shard1_replica_n2/data/]
[junit4] 2> 711983 INFO (qtp1557452267-4041) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard2 r:core_node11 x:halfdeletedcollection2_shard2_replica_n8] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 711983 INFO (qtp1557452267-4041) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard2 r:core_node11 x:halfdeletedcollection2_shard2_replica_n8] o.a.s.c.CoreContainer Creating SolrCore 'halfdeletedcollection2_shard2_replica_n8' using configuration from collection halfdeletedcollection2, trusted=true
[junit4] 2> 711984 INFO (qtp1557452267-4041) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard2 r:core_node11 x:halfdeletedcollection2_shard2_replica_n8] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35809.solr.core.halfdeletedcollection2.shard2.replica_n8' (registry 'solr.core.halfdeletedcollection2.shard2.replica_n8') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@63475939
[junit4] 2> 711984 INFO (qtp1557452267-4041) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard2 r:core_node11 x:halfdeletedcollection2_shard2_replica_n8] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 711984 INFO (qtp1557452267-4041) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard2 r:core_node11 x:halfdeletedcollection2_shard2_replica_n8] o.a.s.c.SolrCore [[halfdeletedcollection2_shard2_replica_n8] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node4/halfdeletedcollection2_shard2_replica_n8], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node4/./halfdeletedcollection2_shard2_replica_n8/data/]
[junit4] 2> 711984 INFO (qtp247979140-4019) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard4 r:core_node24 x:halfdeletedcollection2_shard4_replica_n22] o.a.s.s.IndexSchema [halfdeletedcollection2_shard4_replica_n22] Schema name=minimal
[junit4] 2> 711984 INFO (qtp903444575-4029) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard3 r:core_node15 x:halfdeletedcollection2_shard3_replica_n12] o.a.s.s.IndexSchema [halfdeletedcollection2_shard3_replica_n12] Schema name=minimal
[junit4] 2> 711984 INFO (qtp1557452267-4028) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard3 r:core_node19 x:halfdeletedcollection2_shard3_replica_n16] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 711984 INFO (qtp1557452267-4028) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard3 r:core_node19 x:halfdeletedcollection2_shard3_replica_n16] o.a.s.c.CoreContainer Creating SolrCore 'halfdeletedcollection2_shard3_replica_n16' using configuration from collection halfdeletedcollection2, trusted=true
[junit4] 2> 711985 INFO (qtp2025042261-4012) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard4 r:core_node21 x:halfdeletedcollection2_shard4_replica_n18] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 711985 INFO (qtp2025042261-4012) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard4 r:core_node21 x:halfdeletedcollection2_shard4_replica_n18] o.a.s.c.CoreContainer Creating SolrCore 'halfdeletedcollection2_shard4_replica_n18' using configuration from collection halfdeletedcollection2, trusted=true
[junit4] 2> 711985 INFO (qtp1557452267-4028) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard3 r:core_node19 x:halfdeletedcollection2_shard3_replica_n16] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35809.solr.core.halfdeletedcollection2.shard3.replica_n16' (registry 'solr.core.halfdeletedcollection2.shard3.replica_n16') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@63475939
[junit4] 2> 711985 INFO (qtp1557452267-4028) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard3 r:core_node19 x:halfdeletedcollection2_shard3_replica_n16] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 711985 INFO (qtp1557452267-4028) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard3 r:core_node19 x:halfdeletedcollection2_shard3_replica_n16] o.a.s.c.SolrCore [[halfdeletedcollection2_shard3_replica_n16] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node4/halfdeletedcollection2_shard3_replica_n16], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node4/./halfdeletedcollection2_shard3_replica_n16/data/]
[junit4] 2> 711986 INFO (qtp2025042261-4012) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard4 r:core_node21 x:halfdeletedcollection2_shard4_replica_n18] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35843.solr.core.halfdeletedcollection2.shard4.replica_n18' (registry 'solr.core.halfdeletedcollection2.shard4.replica_n18') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@63475939
[junit4] 2> 711986 INFO (qtp2025042261-4012) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard4 r:core_node21 x:halfdeletedcollection2_shard4_replica_n18] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 711986 INFO (qtp2025042261-4012) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard4 r:core_node21 x:halfdeletedcollection2_shard4_replica_n18] o.a.s.c.SolrCore [[halfdeletedcollection2_shard4_replica_n18] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node1/halfdeletedcollection2_shard4_replica_n18], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node1/./halfdeletedcollection2_shard4_replica_n18/data/]
[junit4] 2> 711986 INFO (qtp247979140-4019) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard4 r:core_node24 x:halfdeletedcollection2_shard4_replica_n22] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 711986 INFO (qtp247979140-4019) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard4 r:core_node24 x:halfdeletedcollection2_shard4_replica_n22] o.a.s.c.CoreContainer Creating SolrCore 'halfdeletedcollection2_shard4_replica_n22' using configuration from collection halfdeletedcollection2, trusted=true
[junit4] 2> 711987 INFO (qtp903444575-4029) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard3 r:core_node15 x:halfdeletedcollection2_shard3_replica_n12] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 711987 INFO (qtp247979140-4019) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard4 r:core_node24 x:halfdeletedcollection2_shard4_replica_n22] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33431.solr.core.halfdeletedcollection2.shard4.replica_n22' (registry 'solr.core.halfdeletedcollection2.shard4.replica_n22') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@63475939
[junit4] 2> 711987 INFO (qtp903444575-4029) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard3 r:core_node15 x:halfdeletedcollection2_shard3_replica_n12] o.a.s.c.CoreContainer Creating SolrCore 'halfdeletedcollection2_shard3_replica_n12' using configuration from collection halfdeletedcollection2, trusted=true
[junit4] 2> 711987 INFO (qtp247979140-4019) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard4 r:core_node24 x:halfdeletedcollection2_shard4_replica_n22] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 711987 INFO (qtp247979140-4019) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard4 r:core_node24 x:halfdeletedcollection2_shard4_replica_n22] o.a.s.c.SolrCore [[halfdeletedcollection2_shard4_replica_n22] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node2/halfdeletedcollection2_shard4_replica_n22], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node2/./halfdeletedcollection2_shard4_replica_n22/data/]
[junit4] 2> 711987 INFO (qtp903444575-4029) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard3 r:core_node15 x:halfdeletedcollection2_shard3_replica_n12] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33849.solr.core.halfdeletedcollection2.shard3.replica_n12' (registry 'solr.core.halfdeletedcollection2.shard3.replica_n12') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@63475939
[junit4] 2> 711987 INFO (qtp903444575-4029) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard3 r:core_node15 x:halfdeletedcollection2_shard3_replica_n12] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 711988 INFO (qtp903444575-4029) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard3 r:core_node15 x:halfdeletedcollection2_shard3_replica_n12] o.a.s.c.SolrCore [[halfdeletedcollection2_shard3_replica_n12] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node3/halfdeletedcollection2_shard3_replica_n12], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node3/./halfdeletedcollection2_shard3_replica_n12/data/]
[junit4] 2> 711988 INFO (qtp903444575-4007) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard4 r:core_node23 x:halfdeletedcollection2_shard4_replica_n20] o.a.s.s.IndexSchema [halfdeletedcollection2_shard4_replica_n20] Schema name=minimal
[junit4] 2> 711988 INFO (qtp247979140-4017) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard3 r:core_node17 x:halfdeletedcollection2_shard3_replica_n14] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 711988 INFO (qtp247979140-4017) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard3 r:core_node17 x:halfdeletedcollection2_shard3_replica_n14] o.a.s.c.CoreContainer Creating SolrCore 'halfdeletedcollection2_shard3_replica_n14' using configuration from collection halfdeletedcollection2, trusted=true
[junit4] 2> 711988 INFO (qtp2025042261-4011) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard2 r:core_node13 x:halfdeletedcollection2_shard2_replica_n10] o.a.s.s.IndexSchema [halfdeletedcollection2_shard2_replica_n10] Schema name=minimal
[junit4] 2> 711988 INFO (qtp247979140-4017) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard3 r:core_node17 x:halfdeletedcollection2_shard3_replica_n14] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33431.solr.core.halfdeletedcollection2.shard3.replica_n14' (registry 'solr.core.halfdeletedcollection2.shard3.replica_n14') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@63475939
[junit4] 2> 711989 INFO (qtp247979140-4017) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard3 r:core_node17 x:halfdeletedcollection2_shard3_replica_n14] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 711989 INFO (qtp247979140-4017) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard3 r:core_node17 x:halfdeletedcollection2_shard3_replica_n14] o.a.s.c.SolrCore [[halfdeletedcollection2_shard3_replica_n14] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node2/halfdeletedcollection2_shard3_replica_n14], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node2/./halfdeletedcollection2_shard3_replica_n14/data/]
[junit4] 2> 711989 INFO (qtp903444575-4007) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard4 r:core_node23 x:halfdeletedcollection2_shard4_replica_n20] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 711989 INFO (qtp903444575-4007) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard4 r:core_node23 x:halfdeletedcollection2_shard4_replica_n20] o.a.s.c.CoreContainer Creating SolrCore 'halfdeletedcollection2_shard4_replica_n20' using configuration from collection halfdeletedcollection2, trusted=true
[junit4] 2> 711989 INFO (qtp903444575-4007) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard4 r:core_node23 x:halfdeletedcollection2_shard4_replica_n20] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33849.solr.core.halfdeletedcollection2.shard4.replica_n20' (registry 'solr.core.halfdeletedcollection2.shard4.replica_n20') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@63475939
[junit4] 2> 711989 INFO (qtp903444575-4007) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard4 r:core_node23 x:halfdeletedcollection2_shard4_replica_n20] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 711989 INFO (qtp903444575-4007) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard4 r:core_node23 x:halfdeletedcollection2_shard4_replica_n20] o.a.s.c.SolrCore [[halfdeletedcollection2_shard4_replica_n20] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node3/halfdeletedcollection2_shard4_replica_n20], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node3/./halfdeletedcollection2_shard4_replica_n20/data/]
[junit4] 2> 711990 INFO (qtp2025042261-4011) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard2 r:core_node13 x:halfdeletedcollection2_shard2_replica_n10] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 711990 INFO (qtp2025042261-4011) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard2 r:core_node13 x:halfdeletedcollection2_shard2_replica_n10] o.a.s.c.CoreContainer Creating SolrCore 'halfdeletedcollection2_shard2_replica_n10' using configuration from collection halfdeletedcollection2, trusted=true
[junit4] 2> 711991 INFO (qtp2025042261-4011) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard2 r:core_node13 x:halfdeletedcollection2_shard2_replica_n10] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35843.solr.core.halfdeletedcollection2.shard2.replica_n10' (registry 'solr.core.halfdeletedcollection2.shard2.replica_n10') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@63475939
[junit4] 2> 711991 INFO (qtp2025042261-4011) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard2 r:core_node13 x:halfdeletedcollection2_shard2_replica_n10] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 711991 INFO (qtp2025042261-4011) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard2 r:core_node13 x:halfdeletedcollection2_shard2_replica_n10] o.a.s.c.SolrCore [[halfdeletedcollection2_shard2_replica_n10] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node1/halfdeletedcollection2_shard2_replica_n10], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001/tempDir-001/node1/./halfdeletedcollection2_shard2_replica_n10/data/]
[junit4] 2> 712094 INFO (qtp1557452267-4041) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard2 r:core_node11 x:halfdeletedcollection2_shard2_replica_n8] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 712094 INFO (qtp1557452267-4041) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard2 r:core_node11 x:halfdeletedcollection2_shard2_replica_n8] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 712094 INFO (qtp903444575-4026) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard1 r:core_node7 x:halfdeletedcollection2_shard1_replica_n4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 712094 INFO (qtp903444575-4026) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard1 r:core_node7 x:halfdeletedcollection2_shard1_replica_n4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 712097 INFO (qtp903444575-4026) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard1 r:core_node7 x:halfdeletedcollection2_shard1_replica_n4] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 712097 INFO (qtp903444575-4026) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard1 r:core_node7 x:halfdeletedcollection2_shard1_replica_n4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 712097 INFO (qtp1557452267-4041) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard2 r:core_node11 x:halfdeletedcollection2_shard2_replica_n8] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 712097 INFO (qtp1557452267-4041) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard2 r:core_node11 x:halfdeletedcollection2_shard2_replica_n8] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 712101 INFO (qtp1557452267-4041) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard2 r:core_node11 x:halfdeletedcollection2_shard2_replica_n8] o.a.s.s.SolrIndexSearcher Opening [Searcher@6f6c8edf[halfdeletedcollection2_shard2_replica_n8] main]
[junit4] 2> 712101 INFO (qtp903444575-4026) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard1 r:core_node7 x:halfdeletedcollection2_shard1_replica_n4] o.a.s.s.SolrIndexSearcher Opening [Searcher@7aee830c[halfdeletedcollection2_shard1_replica_n4] main]
[junit4] 2> 712102 INFO (qtp1557452267-4039) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard1 r:core_node3 x:halfdeletedcollection2_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 712102 INFO (qtp1557452267-4039) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard1 r:core_node3 x:halfdeletedcollection2_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 712102 INFO (qtp1557452267-4041) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard2 r:core_node11 x:halfdeletedcollection2_shard2_replica_n8] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 712102 INFO (qtp903444575-4026) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard1 r:core_node7 x:halfdeletedcollection2_shard1_replica_n4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 712102 INFO (qtp903444575-4026) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard1 r:core_node7 x:halfdeletedcollection2_shard1_replica_n4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 712102 INFO (qtp1557452267-4041) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard2 r:core_node11 x:halfdeletedcollection2_shard2_replica_n8] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 712104 INFO (qtp1557452267-4039) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard1 r:core_node3 x:halfdeletedcollection2_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 712104 INFO (qtp1557452267-4039) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard1 r:core_node3 x:halfdeletedcollection2_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 712104 INFO (qtp903444575-4026) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard1 r:core_node7 x:halfdeletedcollection2_shard1_replica_n4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 712104 INFO (qtp1557452267-4041) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard2 r:core_node11 x:halfdeletedcollection2_shard2_replica_n8] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 712107 INFO (searcherExecutor-1439-thread-1-processing-n:127.0.0.1:35809_solr x:halfdeletedcollection2_shard2_replica_n8 s:shard2 c:halfdeletedcollection2 r:core_node11) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard2 r:core_node11 x:halfdeletedcollection2_shard2_replica_n8] o.a.s.c.SolrCore [halfdeletedcollection2_shard2_replica_n8] Registered new searcher Searcher@6f6c8edf[halfdeletedcollection2_shard2_replica_n8] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 712107 INFO (qtp1557452267-4041) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard2 r:core_node11 x:halfdeletedcollection2_shard2_replica_n8] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1586244682198810624
[junit4] 2> 712107 INFO (searcherExecutor-1436-thread-1-processing-n:127.0.0.1:33849_solr x:halfdeletedcollection2_shard1_replica_n4 s:shard1 c:halfdeletedcollection2 r:core_node7) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard1 r:core_node7 x:halfdeletedcollection2_shard1_replica_n4] o.a.s.c.SolrCore [halfdeletedcollection2_shard1_replica_n4] Registered new searcher Searcher@7aee830c[halfdeletedcollection2_shard1_replica_n4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 712107 INFO (qtp903444575-4026) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard1 r:core_node7 x:halfdeletedcollection2_shard1_replica_n4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1586244682198810624
[junit4] 2> 712110 INFO (qtp1557452267-4039) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard1 r:core_node3 x:halfdeletedcollection2_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@26925704[halfdeletedcollection2_shard1_replica_n1] main]
[junit4] 2> 712110 INFO (qtp247979140-4034) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard2 r:core_node9 x:halfdeletedcollection2_shard2_replica_n6] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 712110 INFO (qtp247979140-4034) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard2 r:core_node9 x:halfdeletedcollection2_shard2_replica_n6] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 712111 INFO (qtp1557452267-4039) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard1 r:core_node3 x:halfdeletedcollection2_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 712111 INFO (qtp2025042261-4008) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard1 r:core_node5 x:halfdeletedcollection2_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 712111 INFO (qtp2025042261-4008) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard1 r:core_node5 x:halfdeletedcollection2_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 712111 INFO (qtp1557452267-4039) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard1 r:core_node3 x:halfdeletedcollection2_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 712111 INFO (qtp903444575-4026) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard1 r:core_node7 x:halfdeletedcollection2_shard1_replica_n4] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=3 found=1 timeoutin=9999ms
[junit4] 2> 712111 INFO (qtp1557452267-4041) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard2 r:core_node11 x:halfdeletedcollection2_shard2_replica_n8] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=3 found=1 timeoutin=9999ms
[junit4] 2> 712113 INFO (qtp2025042261-4008) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard1 r:core_node5 x:halfdeletedcollection2_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 712113 INFO (qtp2025042261-4008) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard1 r:core_node5 x:halfdeletedcollection2_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 712113 INFO (qtp247979140-4034) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard2 r:core_node9 x:halfdeletedcollection2_shard2_replica_n6] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 712113 INFO (qtp247979140-4034) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard2 r:core_node9 x:halfdeletedcollection2_shard2_replica_n6] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 712113 INFO (qtp1557452267-4039) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard1 r:core_node3 x:halfdeletedcollection2_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 712143 INFO (searcherExecutor-1438-thread-1-processing-n:127.0.0.1:35809_solr x:halfdeletedcollection2_shard1_replica_n1 s:shard1 c:halfdeletedcollection2 r:core_node3) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard1 r:core_node3 x:halfdeletedcollection2_shard1_replica_n1] o.a.s.c.SolrCore [halfdeletedcollection2_shard1_replica_n1] Registered new searcher Searcher@26925704[halfdeletedcollection2_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 712143 INFO (qtp1557452267-4039) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard1 r:core_node3 x:halfdeletedcollection2_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1586244682236559360
[junit4] 2> 712143 INFO (qtp247979140-4034) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard2 r:core_node9 x:halfdeletedcollection2_shard2_replica_n6] o.a.s.s.SolrIndexSearcher Opening [Searcher@7dd505a3[halfdeletedcollection2_shard2_replica_n6] main]
[junit4] 2> 712143 INFO (qtp2025042261-4008) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard1 r:core_node5 x:halfdeletedcollection2_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@66709108[halfdeletedcollection2_shard1_replica_n2] main]
[junit4] 2> 712144 INFO (qtp2025042261-4008) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard1 r:core_node5 x:halfdeletedcollection2_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 712144 INFO (qtp2025042261-4008) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard1 r:core_node5 x:halfdeletedcollection2_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 712144 INFO (qtp247979140-4034) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard2 r:core_node9 x:halfdeletedcollection2_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 712145 INFO (qtp247979140-4034) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard2 r:core_node9 x:halfdeletedcollection2_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 712148 INFO (qtp2025042261-4008) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard1 r:core_node5 x:halfdeletedcollection2_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 712148 INFO (qtp247979140-4034) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard2 r:core_node9 x:halfdeletedcollection2_shard2_replica_n6] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 712149 INFO (qtp247979140-4019) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard4 r:core_node24 x:halfdeletedcollection2_shard4_replica_n22] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 712149 INFO (searcherExecutor-1437-thread-1-processing-n:127.0.0.1:35843_solr x:halfdeletedcollection2_shard1_replica_n2 s:shard1 c:halfdeletedcollection2 r:core_node5) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard1 r:core_node5 x:halfdeletedcollection2_shard1_replica_n2] o.a.s.c.SolrCore [halfdeletedcollection2_shard1_replica_n2] Registered new searcher Searcher@66709108[halfdeletedcollection2_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 712149 INFO (qtp247979140-4019) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard4 r:core_node24 x:halfdeletedcollection2_shard4_replica_n22] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 712149 INFO (qtp2025042261-4008) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard1 r:core_node5 x:halfdeletedcollection2_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1586244682242850816
[junit4] 2> 712149 INFO (searcherExecutor-1435-thread-1-processing-n:127.0.0.1:33431_solr x:halfdeletedcollection2_shard2_replica_n6 s:shard2 c:halfdeletedcollection2 r:core_node9) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard2 r:core_node9 x:halfdeletedcollection2_shard2_replica_n6] o.a.s.c.SolrCore [halfdeletedcollection2_shard2_replica_n6] Registered new searcher Searcher@7dd505a3[halfdeletedcollection2_shard2_replica_n6] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 712149 INFO (qtp247979140-4034) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard2 r:core_node9 x:halfdeletedcollection2_shard2_replica_n6] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1586244682242850816
[junit4] 2> 712150 INFO (qtp247979140-4019) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard4 r:core_node24 x:halfdeletedcollection2_shard4_replica_n22] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 712150 INFO (qtp247979140-4019) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard4 r:core_node24 x:halfdeletedcollection2_shard4_replica_n22] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 712151 INFO (qtp247979140-4019) [n:127.0.0.1:33431_solr c:halfdeletedcollection2 s:shard4 r:core_node24 x:halfdeletedcollection2_shard4_replica_n22] o.a.s.s.SolrIndexSearcher Opening [Searcher@54330b7b[halfdeletedcollection2_shard4_replica_n22] main]
[junit4] 2> 712154 INFO (qtp903444575-4007) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard4 r:core_node23 x:halfdeletedcollection2_shard4_replica_n20] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 712154 INFO (qtp2025042261-4011) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard2 r:core_node13 x:halfdeletedcollection2_shard2_replica_n10] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 712154 INFO (qtp903444575-4029) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard3 r:core_node15 x:halfdeletedcollection2_shard3_replica_n12] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 712154 INFO (qtp2025042261-4011) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard2 r:core_node13 x:halfdeletedcollection2_shard2_replica_n10] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 712154 INFO (qtp903444575-4029) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard3 r:core_node15 x:halfdeletedcollection2_shard3_replica_n12] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 712154 INFO (qtp903444575-4007) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard4 r:core_node23 x:halfdeletedcollection2_shard4_replica_n20] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 712155 INFO (qtp903444575-4029) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard3 r:core_node15 x:halfdeletedcollection2_shard3_replica_n12] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 712155 INFO (qtp2025042261-4011) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard2 r:core_node13 x:halfdeletedcollection2_shard2_replica_n10] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 712155 INFO (qtp903444575-4029) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard3 r:core_node15 x:halfdeletedcollection2_shard3_replica_n12] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 712155 INFO (qtp2025042261-4011) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard2 r:core_node13 x:halfdeletedcollection2_shard2_replica_n10] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 712155 INFO (qtp903444575-4007) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard4 r:core_node23 x:halfdeletedcollection2_shard4_replica_n20] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 712155 INFO (qtp903444575-4007) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard4 r:core_node23 x:halfdeletedcollection2_shard4_replica_n20] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 712155 INFO (qtp1557452267-4028) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard3 r:core_node19 x:halfdeletedcollection2_shard3_replica_n16] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 712155 INFO (qtp1557452267-4028) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard3 r:core_node19 x:halfdeletedcollection2_shard3_replica_n16] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 712156 INFO (qtp1557452267-4028) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard3 r:core_node19 x:halfdeletedcollection2_shard3_replica_n16] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 712156 INFO (qtp1557452267-4028) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard3 r:core_node19 x:halfdeletedcollection2_shard3_replica_n16] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 712156 INFO (qtp903444575-4029) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard3 r:core_node15 x:halfdeletedcollection2_shard3_replica_n12] o.a.s.s.SolrIndexSearcher Opening [Searcher@1af0229c[halfdeletedcollection2_shard3_replica_n12] main]
[junit4] 2> 712156 INFO (qtp2025042261-4011) [n:127.0.0.1:35843_solr c:halfdeletedcollection2 s:shard2 r:core_node13 x:halfdeletedcollection2_shard2_replica_n10] o.a.s.s.SolrIndexSearcher Opening [Searcher@3af83165[halfdeletedcollection2_shard2_replica_n10] main]
[junit4] 2> 712157 INFO (qtp903444575-4007) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard4 r:core_node23 x:halfdeletedcollection2_shard4_replica_n20] o.a.s.s.SolrIndexSearcher Opening [Searcher@165fe89b[halfdeletedcollection2_shard4_replica_n20] main]
[junit4] 2> 712157 INFO (qtp1557452267-4028) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard3 r:core_node19 x:halfdeletedcollection2_shard3_replica_n16] o.a.s.s.SolrIndexSearcher Opening [Searcher@68217069[halfdeletedcollection2_shard3_replica_n16] main]
[junit4] 2> 712158 INFO (qtp1557452267-4028) [n:127.0.0.1:35809_solr c:halfdeletedcollection2 s:shard3 r:core_node19 x:halfdeletedcollection2_shard3_replica_n16] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 712158 INFO (qtp903444575-4029) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard3 r:core_node15 x:halfdeletedcollection2_shard3_replica_n12] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 712158 INFO (qtp903444575-4007) [n:127.0.0.1:338
[...truncated too long message...]
2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 920671 ERROR (qtp903444575-4029) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard3 r:core_node15 x:halfdeletedcollection2_shard3_replica_n12] o.a.s.c.ZkContainer :org.apache.solr.common.SolrException: Error getting leader from zk for shard shard3
[junit4] 2> at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1207)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1079)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1014)
[junit4] 2> at org.apache.solr.core.ZkContainer.lambda$registerInZk$0(ZkContainer.java:181)
[junit4] 2> at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:208)
[junit4] 2> at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:886)
[junit4] 2> at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1047)
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:948)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:735)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:716)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:497)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: org.apache.solr.common.SolrException: No registered leader was found after waiting for 1220000ms , collection: halfdeletedcollection2 slice: shard3 saw state=null with live_nodes=[]
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:772)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:737)
[junit4] 2> at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1178)
[junit4] 2> ... 41 more
[junit4] 2>
[junit4] 2> 920677 INFO (qtp903444575-4029) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard3 r:core_node15 x:halfdeletedcollection2_shard3_replica_n12] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node15&collection.configName=conf&newCollection=true&name=halfdeletedcollection2_shard3_replica_n12&action=CREATE&numShards=4&collection=halfdeletedcollection2&shard=shard3&wt=javabin&version=2&replicaType=NRT} status=0 QTime=209770
[junit4] 2> 920677 INFO (qtp903444575-4007) [n:127.0.0.1:33849_solr c:halfdeletedcollection2 s:shard4 r:core_node23 x:halfdeletedcollection2_shard4_replica_n20] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node23&collection.configName=conf&newCollection=true&name=halfdeletedcollection2_shard4_replica_n20&action=CREATE&numShards=4&collection=halfdeletedcollection2&shard=shard4&wt=javabin&version=2&replicaType=NRT} status=0 QTime=209768
[junit4] 2> 920677 ERROR (SUITE-TestDeleteCollectionOnDownNodes-seed#[14BB2B640D0472EB]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 920678 INFO (SUITE-TestDeleteCollectionOnDownNodes-seed#[14BB2B640D0472EB]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41153 41153
[junit4] 2> 922233 INFO (Thread-680) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41153 41153
[junit4] 2> 922234 WARN (Thread-680) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/aliases.json
[junit4] 2> 5 /solr/clusterprops.json
[junit4] 2> 4 /solr/security.json
[junit4] 2> 4 /solr/configs/conf
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 12 /solr/collections/halfdeletedcollection2/state.json
[junit4] 2> 5 /solr/clusterstate.json
[junit4] 2> 2 /solr/collections/halfdeletedcollection2/leader_elect/shard2/election/99140292504387595-core_node11-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/live_nodes
[junit4] 2> 5 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestDeleteCollectionOnDownNodes_14BB2B640D0472EB-001
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=false): {}, locale=mt, timezone=Zulu
[junit4] 2> NOTE: Linux 4.10.0-40-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=8,threads=1,free=167803744,total=518979584
[junit4] 2> NOTE: All tests run in this JVM: [TestClusterStateMutator, TestPolicyCloud, ChangedSchemaMergeTest, ZkFailoverTest, TestSlowCompositeReaderWrapper, NodeLostTriggerTest, PKIAuthenticationIntegrationTest, ShardSplitTest, PrimUtilsTest, LegacyCloudClusterPropTest, TestClassicSimilarityFactory, TestPseudoReturnFields, CoreAdminCreateDiscoverTest, OpenCloseCoreStressTest, TestCursorMarkWithoutUniqueKey, TestCopyFieldCollectionResource, CircularListTest, TestSolrCLIRunExample, TestRandomCollapseQParserPlugin, TestDeleteCollectionOnDownNodes]
[junit4] Completed [209/751 (1!)] on J1 in 212.18s, 1 test, 1 error <<< FAILURES!
[...truncated 49125 lines...]
[JENKINS] Lucene-Solr-7.x-Linux (64bit/jdk1.8.0_144) - Build # 959
- Still unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/959/
Java: 64bit/jdk1.8.0_144 -XX:-UseCompressedOops -XX:+UseParallelGC
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.client.solrj.io.graph.GraphTest
Error Message:
Error from server at https://127.0.0.1:36255/solr: create the collection time out:180s
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:36255/solr: create the collection time out:180s
at __randomizedtesting.SeedInfo.seed([8F89C5528A40B32]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1103)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:883)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.client.solrj.io.graph.GraphTest.setupCluster(GraphTest.java:62)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:874)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToPullReplica
Error Message:
Error from server at http://127.0.0.1:35227/solr: Could not fully create collection: pull_replica_error_handling_test_cant_connect_to_pull_replica
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:35227/solr: Could not fully create collection: pull_replica_error_handling_test_cant_connect_to_pull_replica
at __randomizedtesting.SeedInfo.seed([FFE982745EB68207:F7DE64BC8518F689]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1103)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:883)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToPullReplica(TestPullReplicaErrorHandling.java:140)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12515 lines...]
[junit4] Suite: org.apache.solr.cloud.TestPullReplicaErrorHandling
[junit4] 2> 770428 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplicaErrorHandling_FFE982745EB68207-001/init-core-data-001
[junit4] 2> 770429 WARN (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 770429 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 770430 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 770430 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplicaErrorHandling_FFE982745EB68207-001/tempDir-001
[junit4] 2> 770430 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 770430 INFO (Thread-2140) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 770430 INFO (Thread-2140) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 770431 ERROR (Thread-2140) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 770530 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:41457
[junit4] 2> 770533 INFO (zkConnectionManagerCallback-1553-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 770537 INFO (jetty-launcher-1550-thread-1) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 770537 INFO (jetty-launcher-1550-thread-3) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 770537 INFO (jetty-launcher-1550-thread-4) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 770538 INFO (jetty-launcher-1550-thread-2) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 770541 INFO (jetty-launcher-1550-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1b7de8bf{/solr,null,AVAILABLE}
[junit4] 2> 770545 INFO (jetty-launcher-1550-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@361abead{/solr,null,AVAILABLE}
[junit4] 2> 770547 INFO (jetty-launcher-1550-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@290f8fa6{/solr,null,AVAILABLE}
[junit4] 2> 770547 INFO (jetty-launcher-1550-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1fbf3287{/solr,null,AVAILABLE}
[junit4] 2> 770549 INFO (jetty-launcher-1550-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@41604ac4{HTTP/1.1,[http/1.1]}{127.0.0.1:44713}
[junit4] 2> 770549 INFO (jetty-launcher-1550-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@30407662{HTTP/1.1,[http/1.1]}{127.0.0.1:37851}
[junit4] 2> 770549 INFO (jetty-launcher-1550-thread-1) [ ] o.e.j.s.Server Started @771979ms
[junit4] 2> 770549 INFO (jetty-launcher-1550-thread-3) [ ] o.e.j.s.Server Started @771979ms
[junit4] 2> 770549 INFO (jetty-launcher-1550-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44713}
[junit4] 2> 770549 INFO (jetty-launcher-1550-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37851}
[junit4] 2> 770550 ERROR (jetty-launcher-1550-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 770550 ERROR (jetty-launcher-1550-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 770550 INFO (jetty-launcher-1550-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 770550 INFO (jetty-launcher-1550-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 770550 INFO (jetty-launcher-1550-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 770550 INFO (jetty-launcher-1550-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 770550 INFO (jetty-launcher-1550-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 770550 INFO (jetty-launcher-1550-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 770550 INFO (jetty-launcher-1550-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-08T15:30:39.369Z
[junit4] 2> 770550 INFO (jetty-launcher-1550-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-08T15:30:39.369Z
[junit4] 2> 770553 INFO (jetty-launcher-1550-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@749f0397{HTTP/1.1,[http/1.1]}{127.0.0.1:41103}
[junit4] 2> 770553 INFO (jetty-launcher-1550-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3ac77319{HTTP/1.1,[http/1.1]}{127.0.0.1:46479}
[junit4] 2> 770553 INFO (jetty-launcher-1550-thread-2) [ ] o.e.j.s.Server Started @771983ms
[junit4] 2> 770553 INFO (jetty-launcher-1550-thread-4) [ ] o.e.j.s.Server Started @771983ms
[junit4] 2> 770553 INFO (jetty-launcher-1550-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41103}
[junit4] 2> 770553 INFO (jetty-launcher-1550-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46479}
[junit4] 2> 770553 ERROR (jetty-launcher-1550-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 770553 ERROR (jetty-launcher-1550-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 770553 INFO (zkConnectionManagerCallback-1556-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 770553 INFO (zkConnectionManagerCallback-1557-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 770553 INFO (jetty-launcher-1550-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 770553 INFO (jetty-launcher-1550-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 770554 INFO (jetty-launcher-1550-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 770554 INFO (jetty-launcher-1550-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 770554 INFO (jetty-launcher-1550-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 770554 INFO (jetty-launcher-1550-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 770554 INFO (jetty-launcher-1550-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-08T15:30:39.373Z
[junit4] 2> 770554 INFO (jetty-launcher-1550-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-08T15:30:39.373Z
[junit4] 2> 770554 INFO (jetty-launcher-1550-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 770554 INFO (jetty-launcher-1550-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 770556 INFO (zkConnectionManagerCallback-1559-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 770557 INFO (jetty-launcher-1550-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 770561 INFO (zkConnectionManagerCallback-1561-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 770562 INFO (jetty-launcher-1550-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 770565 INFO (jetty-launcher-1550-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41457/solr
[junit4] 2> 770565 INFO (jetty-launcher-1550-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41457/solr
[junit4] 2> 770569 INFO (zkConnectionManagerCallback-1569-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 770569 INFO (zkConnectionManagerCallback-1567-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 770570 INFO (jetty-launcher-1550-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41457/solr
[junit4] 2> 770573 INFO (zkConnectionManagerCallback-1575-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 770574 INFO (zkConnectionManagerCallback-1577-thread-1-processing-n:127.0.0.1:44713_solr) [n:127.0.0.1:44713_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 770574 INFO (jetty-launcher-1550-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41457/solr
[junit4] 2> 770574 INFO (zkConnectionManagerCallback-1573-thread-1-processing-n:127.0.0.1:37851_solr) [n:127.0.0.1:37851_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 770575 INFO (zkConnectionManagerCallback-1581-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 770575 INFO (zkConnectionManagerCallback-1583-thread-1-processing-n:127.0.0.1:41103_solr) [n:127.0.0.1:41103_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 770577 INFO (zkConnectionManagerCallback-1585-thread-1-processing-n:127.0.0.1:46479_solr) [n:127.0.0.1:46479_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 770616 INFO (jetty-launcher-1550-thread-3) [n:127.0.0.1:37851_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 770616 INFO (jetty-launcher-1550-thread-1) [n:127.0.0.1:44713_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 770616 INFO (jetty-launcher-1550-thread-4) [n:127.0.0.1:46479_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 770616 INFO (jetty-launcher-1550-thread-3) [n:127.0.0.1:37851_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37851_solr
[junit4] 2> 770616 INFO (jetty-launcher-1550-thread-1) [n:127.0.0.1:44713_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44713_solr
[junit4] 2> 770616 INFO (jetty-launcher-1550-thread-4) [n:127.0.0.1:46479_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46479_solr
[junit4] 2> 770616 INFO (jetty-launcher-1550-thread-3) [n:127.0.0.1:37851_solr ] o.a.s.c.Overseer Overseer (id=99139389964288007-127.0.0.1:37851_solr-n_0000000000) starting
[junit4] 2> 770621 INFO (zkCallback-1572-thread-1-processing-n:127.0.0.1:37851_solr) [n:127.0.0.1:37851_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 770621 INFO (zkCallback-1584-thread-1-processing-n:127.0.0.1:46479_solr) [n:127.0.0.1:46479_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 770622 INFO (zkCallback-1576-thread-1-processing-n:127.0.0.1:44713_solr) [n:127.0.0.1:44713_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 770622 INFO (jetty-launcher-1550-thread-2) [n:127.0.0.1:41103_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 770623 INFO (jetty-launcher-1550-thread-2) [n:127.0.0.1:41103_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 770623 INFO (jetty-launcher-1550-thread-2) [n:127.0.0.1:41103_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41103_solr
[junit4] 2> 770629 INFO (zkCallback-1572-thread-1-processing-n:127.0.0.1:37851_solr) [n:127.0.0.1:37851_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 770629 INFO (zkCallback-1584-thread-1-processing-n:127.0.0.1:46479_solr) [n:127.0.0.1:46479_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 770629 INFO (zkCallback-1576-thread-1-processing-n:127.0.0.1:44713_solr) [n:127.0.0.1:44713_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 770633 INFO (zkCallback-1582-thread-1-processing-n:127.0.0.1:41103_solr) [n:127.0.0.1:41103_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 770633 INFO (jetty-launcher-1550-thread-3) [n:127.0.0.1:37851_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37851_solr
[junit4] 2> 770634 INFO (zkCallback-1576-thread-1-processing-n:127.0.0.1:44713_solr) [n:127.0.0.1:44713_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 770634 INFO (zkCallback-1582-thread-1-processing-n:127.0.0.1:41103_solr) [n:127.0.0.1:41103_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 770634 INFO (zkCallback-1584-thread-1-processing-n:127.0.0.1:46479_solr) [n:127.0.0.1:46479_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 770647 INFO (zkCallback-1572-thread-1-processing-n:127.0.0.1:37851_solr) [n:127.0.0.1:37851_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 770660 INFO (jetty-launcher-1550-thread-2) [n:127.0.0.1:41103_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41103.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@336c09cf
[junit4] 2> 770669 INFO (jetty-launcher-1550-thread-2) [n:127.0.0.1:41103_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41103.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@336c09cf
[junit4] 2> 770669 INFO (jetty-launcher-1550-thread-2) [n:127.0.0.1:41103_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41103.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@336c09cf
[junit4] 2> 770669 INFO (jetty-launcher-1550-thread-2) [n:127.0.0.1:41103_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplicaErrorHandling_FFE982745EB68207-001/tempDir-001/node2/.
[junit4] 2> 770671 INFO (jetty-launcher-1550-thread-4) [n:127.0.0.1:46479_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46479.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@336c09cf
[junit4] 2> 770675 INFO (jetty-launcher-1550-thread-4) [n:127.0.0.1:46479_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46479.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@336c09cf
[junit4] 2> 770676 INFO (jetty-launcher-1550-thread-4) [n:127.0.0.1:46479_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46479.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@336c09cf
[junit4] 2> 770676 INFO (jetty-launcher-1550-thread-4) [n:127.0.0.1:46479_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplicaErrorHandling_FFE982745EB68207-001/tempDir-001/node4/.
[junit4] 2> 770684 INFO (jetty-launcher-1550-thread-3) [n:127.0.0.1:37851_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37851.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@336c09cf
[junit4] 2> 770686 INFO (jetty-launcher-1550-thread-1) [n:127.0.0.1:44713_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44713.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@336c09cf
[junit4] 2> 770691 INFO (jetty-launcher-1550-thread-1) [n:127.0.0.1:44713_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44713.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@336c09cf
[junit4] 2> 770691 INFO (jetty-launcher-1550-thread-3) [n:127.0.0.1:37851_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37851.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@336c09cf
[junit4] 2> 770691 INFO (jetty-launcher-1550-thread-1) [n:127.0.0.1:44713_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44713.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@336c09cf
[junit4] 2> 770691 INFO (jetty-launcher-1550-thread-3) [n:127.0.0.1:37851_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37851.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@336c09cf
[junit4] 2> 770692 INFO (jetty-launcher-1550-thread-1) [n:127.0.0.1:44713_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplicaErrorHandling_FFE982745EB68207-001/tempDir-001/node1/.
[junit4] 2> 770692 INFO (jetty-launcher-1550-thread-3) [n:127.0.0.1:37851_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplicaErrorHandling_FFE982745EB68207-001/tempDir-001/node3/.
[junit4] 2> 770746 INFO (zkConnectionManagerCallback-1592-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 770749 INFO (zkConnectionManagerCallback-1596-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 770750 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 770750 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41457/solr ready
[junit4] 2> 770752 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@749f0397{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 770752 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=5244231
[junit4] 2> 770753 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 770753 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5a1c8e22: rootName = solr_41103, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@73e3ab41
[junit4] 2> 770756 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 770756 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6742f654: rootName = solr_41103, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@5c162449
[junit4] 2> 770758 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 770758 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@ee1929f: rootName = solr_41103, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@399f46cf
[junit4] 2> 770759 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:41103_solr
[junit4] 2> 770759 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 770759 INFO (zkCallback-1576-thread-1-processing-n:127.0.0.1:44713_solr) [n:127.0.0.1:44713_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 770759 INFO (zkCallback-1582-thread-1-processing-n:127.0.0.1:41103_solr) [n:127.0.0.1:41103_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 770759 INFO (zkCallback-1584-thread-1-processing-n:127.0.0.1:46479_solr) [n:127.0.0.1:46479_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 770759 INFO (zkCallback-1572-thread-1-processing-n:127.0.0.1:37851_solr) [n:127.0.0.1:37851_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 770760 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 770760 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1fbf3287{/solr,null,UNAVAILABLE}
[junit4] 2> 770761 INFO (zkCallback-1595-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 770761 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 770761 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@332f5530{/solr,null,AVAILABLE}
[junit4] 2> 770761 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.e.j.s.AbstractConnector Started ServerConnector@39e3601a{HTTP/1.1,[http/1.1]}{127.0.0.1:44689}
[junit4] 2> 770761 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.e.j.s.Server Started @772191ms
[junit4] 2> 770761 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36783}
[junit4] 2> 770762 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 770762 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 770762 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 770762 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 770762 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-08T15:30:39.581Z
[junit4] 2> 770765 INFO (zkConnectionManagerCallback-1598-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 770766 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 770770 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41457/solr
[junit4] 2> 770771 INFO (zkConnectionManagerCallback-1602-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 770772 INFO (zkConnectionManagerCallback-1604-thread-1-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 770774 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:36783_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 770774 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:36783_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 770775 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:36783_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36783_solr
[junit4] 2> 770775 INFO (zkCallback-1572-thread-1-processing-n:127.0.0.1:37851_solr) [n:127.0.0.1:37851_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 770775 INFO (zkCallback-1576-thread-1-processing-n:127.0.0.1:44713_solr) [n:127.0.0.1:44713_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 770775 INFO (zkCallback-1595-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 770775 INFO (zkCallback-1584-thread-1-processing-n:127.0.0.1:46479_solr) [n:127.0.0.1:46479_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 770777 INFO (zkCallback-1603-thread-1-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 770809 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:36783_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36783.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@336c09cf
[junit4] 2> 770813 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:36783_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36783.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@336c09cf
[junit4] 2> 770813 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:36783_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36783.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@336c09cf
[junit4] 2> 770814 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:36783_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplicaErrorHandling_FFE982745EB68207-001/tempDir-001/node2/.
[junit4] 2> 770837 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.TestPullReplicaErrorHandling Adding proxy for URL: http://127.0.0.1:44689/solr. Proxy: http://127.0.0.1:36783/solr
[junit4] 2> 770838 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@30407662{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 770838 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=53263480
[junit4] 2> 770838 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 770838 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4eb5934: rootName = solr_37851, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@1d919e95
[junit4] 2> 770842 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 770842 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@db719b8: rootName = solr_37851, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@5c162449
[junit4] 2> 770844 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 770844 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@dce8706: rootName = solr_37851, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@399f46cf
[junit4] 2> 770845 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:37851_solr
[junit4] 2> 770845 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 770845 INFO (zkCallback-1572-thread-1-processing-n:127.0.0.1:37851_solr) [n:127.0.0.1:37851_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 770845 INFO (zkCallback-1595-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 770845 INFO (zkCallback-1584-thread-1-processing-n:127.0.0.1:46479_solr) [n:127.0.0.1:46479_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 770845 INFO (zkCallback-1603-thread-1-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 770845 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 770845 INFO (zkCallback-1576-thread-1-processing-n:127.0.0.1:44713_solr) [n:127.0.0.1:44713_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 770846 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.Overseer Overseer (id=99139389964288007-127.0.0.1:37851_solr-n_0000000000) closing
[junit4] 2> 770846 INFO (OverseerStateUpdate-99139389964288007-127.0.0.1:37851_solr-n_0000000000) [n:127.0.0.1:37851_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:37851_solr
[junit4] 2> 770846 WARN (OverseerAutoScalingTriggerThread-99139389964288007-127.0.0.1:37851_solr-n_0000000000) [n:127.0.0.1:37851_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 770849 INFO (zkCallback-1576-thread-1-processing-n:127.0.0.1:44713_solr) [n:127.0.0.1:44713_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44713_solr
[junit4] 2> 770849 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1b7de8bf{/solr,null,UNAVAILABLE}
[junit4] 2> 770849 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 770849 INFO (zkCallback-1576-thread-1-processing-n:127.0.0.1:44713_solr) [n:127.0.0.1:44713_solr ] o.a.s.c.Overseer Overseer (id=99139389964288009-127.0.0.1:44713_solr-n_0000000001) starting
[junit4] 2> 770850 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@10e9edc{/solr,null,AVAILABLE}
[junit4] 2> 770850 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5c425c5a{HTTP/1.1,[http/1.1]}{127.0.0.1:36823}
[junit4] 2> 770850 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.e.j.s.Server Started @772280ms
[junit4] 2> 770850 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46741}
[junit4] 2> 770850 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 770850 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 770850 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 770851 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 770851 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-08T15:30:39.670Z
[junit4] 2> 770853 INFO (zkConnectionManagerCallback-1607-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 770853 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 770866 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41457/solr
[junit4] 2> 770867 INFO (zkConnectionManagerCallback-1612-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 770868 INFO (zkConnectionManagerCallback-1614-thread-1-processing-n:127.0.0.1:46741_solr) [n:127.0.0.1:46741_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 770871 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:46741_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 770872 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:46741_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 770872 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:46741_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46741_solr
[junit4] 2> 770873 INFO (zkCallback-1584-thread-1-processing-n:127.0.0.1:46479_solr) [n:127.0.0.1:46479_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 770873 INFO (zkCallback-1576-thread-1-processing-n:127.0.0.1:44713_solr) [n:127.0.0.1:44713_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 770873 INFO (zkCallback-1595-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 770873 INFO (zkCallback-1603-thread-1-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 770873 INFO (zkCallback-1613-thread-1-processing-n:127.0.0.1:46741_solr) [n:127.0.0.1:46741_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 770925 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:46741_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46741.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@336c09cf
[junit4] 2> 770930 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:46741_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46741.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@336c09cf
[junit4] 2> 770930 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:46741_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46741.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@336c09cf
[junit4] 2> 770931 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:46741_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplicaErrorHandling_FFE982745EB68207-001/tempDir-001/node3/.
[junit4] 2> 770953 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.TestPullReplicaErrorHandling Adding proxy for URL: http://127.0.0.1:36823/solr. Proxy: http://127.0.0.1:46741/solr
[junit4] 2> 770954 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@3ac77319{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 770954 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=68268094
[junit4] 2> 770954 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 770954 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1e98b493: rootName = solr_46479, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@69bda45e
[junit4] 2> 770957 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 770957 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5a3d5ad8: rootName = solr_46479, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@5c162449
[junit4] 2> 770960 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 770960 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@16e364db: rootName = solr_46479, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@399f46cf
[junit4] 2> 770960 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:46479_solr
[junit4] 2> 770961 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 770961 INFO (zkCallback-1584-thread-1-processing-n:127.0.0.1:46479_solr) [n:127.0.0.1:46479_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 770961 INFO (zkCallback-1576-thread-1-processing-n:127.0.0.1:44713_solr) [n:127.0.0.1:44713_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 770961 INFO (zkCallback-1603-thread-1-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 770961 INFO (zkCallback-1613-thread-1-processing-n:127.0.0.1:46741_solr) [n:127.0.0.1:46741_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 770961 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 770961 INFO (zkCallback-1595-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 770961 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@290f8fa6{/solr,null,UNAVAILABLE}
[junit4] 2> 770962 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 770962 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6de92e03{/solr,null,AVAILABLE}
[junit4] 2> 770962 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.e.j.s.AbstractConnector Started ServerConnector@453b044c{HTTP/1.1,[http/1.1]}{127.0.0.1:41461}
[junit4] 2> 770963 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.e.j.s.Server Started @772393ms
[junit4] 2> 770963 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35227}
[junit4] 2> 770963 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 770963 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 770963 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 770963 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 770963 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-08T15:30:39.782Z
[junit4] 2> 770964 INFO (zkConnectionManagerCallback-1617-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 770965 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 770969 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41457/solr
[junit4] 2> 770969 INFO (zkConnectionManagerCallback-1621-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 770970 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x16036bffa120016, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 770970 INFO (zkConnectionManagerCallback-1623-thread-1-processing-n:127.0.0.1:35227_solr) [n:127.0.0.1:35227_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 770972 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:35227_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 770973 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:35227_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 770973 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:35227_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35227_solr
[junit4] 2> 770974 INFO (zkCallback-1613-thread-1-processing-n:127.0.0.1:46741_solr) [n:127.0.0.1:46741_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 770974 INFO (zkCallback-1595-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 770974 INFO (zkCallback-1603-thread-1-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 770974 INFO (zkCallback-1576-thread-1-processing-n:127.0.0.1:44713_solr) [n:127.0.0.1:44713_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 770974 INFO (zkCallback-1622-thread-1-processing-n:127.0.0.1:35227_solr) [n:127.0.0.1:35227_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 771007 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:35227_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35227.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@336c09cf
[junit4] 2> 771011 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:35227_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35227.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@336c09cf
[junit4] 2> 771011 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:35227_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35227.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@336c09cf
[junit4] 2> 771012 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:35227_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplicaErrorHandling_FFE982745EB68207-001/tempDir-001/node4/.
[junit4] 2> 771033 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.TestPullReplicaErrorHandling Adding proxy for URL: http://127.0.0.1:41461/solr. Proxy: http://127.0.0.1:35227/solr
[junit4] 2> 771033 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@41604ac4{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 771033 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1907578825
[junit4] 2> 771033 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 771033 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@177aedc2: rootName = solr_44713, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@7edfd448
[junit4] 2> 771036 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 771036 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3c2b7b85: rootName = solr_44713, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@5c162449
[junit4] 2> 771039 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 771039 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5776b0ec: rootName = solr_44713, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@399f46cf
[junit4] 2> 771040 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:44713_solr
[junit4] 2> 771040 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 771040 INFO (zkCallback-1595-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 771040 INFO (zkCallback-1576-thread-1-processing-n:127.0.0.1:44713_solr) [n:127.0.0.1:44713_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 771040 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 771040 INFO (zkCallback-1603-thread-1-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 771040 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.Overseer Overseer (id=99139389964288009-127.0.0.1:44713_solr-n_0000000001) closing
[junit4] 2> 771041 INFO (OverseerStateUpdate-99139389964288009-127.0.0.1:44713_solr-n_0000000001) [n:127.0.0.1:44713_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:44713_solr
[junit4] 2> 771041 WARN (OverseerAutoScalingTriggerThread-99139389964288009-127.0.0.1:44713_solr-n_0000000001) [n:127.0.0.1:44713_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 771042 INFO (zkCallback-1622-thread-1-processing-n:127.0.0.1:35227_solr) [n:127.0.0.1:35227_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 771042 INFO (zkCallback-1613-thread-1-processing-n:127.0.0.1:46741_solr) [n:127.0.0.1:46741_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 771043 INFO (zkCallback-1603-thread-2-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36783_solr
[junit4] 2> 771043 INFO (zkCallback-1603-thread-2-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr ] o.a.s.c.Overseer Overseer (id=99139389964288017-127.0.0.1:36783_solr-n_0000000004) starting
[junit4] 2> 772542 WARN (zkCallback-1576-thread-1-processing-n:127.0.0.1:44713_solr) [n:127.0.0.1:44713_solr ] o.a.s.c.ZkController Unable to register nodeLost path for 127.0.0.1:44713_solr
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling/nodeLost/127.0.0.1:44713_solr
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$create$7(SolrZkClient.java:364)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:364)
[junit4] 2> at org.apache.solr.cloud.ZkController.lambda$registerLiveNodesListener$2(ZkController.java:879)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.lambda$refreshLiveNodes$1(ZkStateReader.java:705)
[junit4] 2> at java.util.concurrent.ConcurrentHashMap$KeySetView.forEach(ConcurrentHashMap.java:4649)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:704)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.access$900(ZkStateReader.java:74)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.refreshAndWatch(ZkStateReader.java:1105)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.process(ZkStateReader.java:1100)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$1.lambda$process$1(SolrZkClient.java:268)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 772543 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@361abead{/solr,null,UNAVAILABLE}
[junit4] 2> 772544 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 772545 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5115d7dd{/solr,null,AVAILABLE}
[junit4] 2> 772546 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3d0c7cd4{HTTP/1.1,[http/1.1]}{127.0.0.1:36527}
[junit4] 2> 772546 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.e.j.s.Server Started @773976ms
[junit4] 2> 772547 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34481}
[junit4] 2> 772547 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 772547 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 772547 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 772547 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 772547 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-08T15:30:41.366Z
[junit4] 2> 772549 INFO (zkConnectionManagerCallback-1627-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 772550 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 772559 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41457/solr
[junit4] 2> 772561 INFO (zkConnectionManagerCallback-1631-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 772563 INFO (zkConnectionManagerCallback-1633-thread-1-processing-n:127.0.0.1:34481_solr) [n:127.0.0.1:34481_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 772568 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:34481_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 772570 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:34481_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 772572 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:34481_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34481_solr
[junit4] 2> 772573 INFO (zkCallback-1595-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 772573 INFO (zkCallback-1613-thread-1-processing-n:127.0.0.1:46741_solr) [n:127.0.0.1:46741_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 772573 INFO (zkCallback-1603-thread-2-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 772574 INFO (zkCallback-1622-thread-1-processing-n:127.0.0.1:35227_solr) [n:127.0.0.1:35227_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 772574 INFO (zkCallback-1632-thread-1-processing-n:127.0.0.1:34481_solr) [n:127.0.0.1:34481_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 772665 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:34481_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34481.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@336c09cf
[junit4] 2> 772670 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:34481_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34481.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@336c09cf
[junit4] 2> 772670 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:34481_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34481.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@336c09cf
[junit4] 2> 772671 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [n:127.0.0.1:34481_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplicaErrorHandling_FFE982745EB68207-001/tempDir-001/node1/.
[junit4] 2> 772691 INFO (SUITE-TestPullReplicaErrorHandling-seed#[FFE982745EB68207]-worker) [ ] o.a.s.c.TestPullReplicaErrorHandling Adding proxy for URL: http://127.0.0.1:36527/solr. Proxy: http://127.0.0.1:34481/solr
[junit4] 2> 772691 INFO (SocketProxy-Acceptor-34481) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=37570,localport=34481], receiveBufferSize:531000
[junit4] 2> 772693 INFO (SocketProxy-Acceptor-34481) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=36527,localport=48776], receiveBufferSize=530904
[junit4] 2> 772694 INFO (qtp1907738058-7695) [n:127.0.0.1:34481_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :clusterprop with params val=false&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 772695 INFO (qtp1907738058-7695) [n:127.0.0.1:34481_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={val=false&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 772710 INFO (TEST-TestPullReplicaErrorHandling.testCantConnectToLeader-seed#[FFE982745EB68207]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCantConnectToLeader
[junit4] 2> 772711 INFO (qtp1907738058-7696) [n:127.0.0.1:34481_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=1&replicationFactor=1&collection.configName=conf&maxShardsPerNode=1&name=pull_replica_error_handling_test_cant_connect_to_leader&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 772713 INFO (OverseerThreadFactory-3496-thread-1-processing-n:127.0.0.1:36783_solr) [n:127.0.0.1:36783_solr ] o.a.s.c.CreateCollectionCmd Create collection pull_replica_error_handling_test_cant_connect_to_leader
[junit4] 2> 772816 INFO (OverseerStateUpdate-99139389964288017-127.0.0.1:36783_solr-n_0000000004) [n:127.0.0.1:36783_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"pull_replica_error_handling_test_cant_connect_to_leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:35227/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 772817 INFO (OverseerStateUpdate-99139389964288017-127.0.0.1:36783_solr-n_0000000004) [n:127.0.0.1:36783_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"pull_replica_error_handling_test_cant_connect_to_leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:34481/solr",
[junit4] 2> "type":"PULL",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 773019 INFO (SocketProxy-Acceptor-35227) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=44698,localport=35227], receiveBufferSize:531000
[junit4] 2> 773019 INFO (SocketProxy-Acceptor-34481) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=37576,localport=34481], receiveBufferSize:531000
[junit4] 2> 773022 INFO (SocketProxy-Acceptor-35227) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=41461,localport=40820], receiveBufferSize=530904
[junit4] 2> 773022 INFO (SocketProxy-Acceptor-34481) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=36527,localport=48784], receiveBufferSize=530904
[junit4] 2> 773026 INFO (qtp901139553-7661) [n:127.0.0.1:35227_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_n1&action=CREATE&numShards=1&collection=pull_replica_error_handling_test_cant_connect_to_leader&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 773026 INFO (qtp1907738058-7690) [n:127.0.0.1:34481_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2&action=CREATE&numShards=1&collection=pull_replica_error_handling_test_cant_connect_to_leader&shard=shard1&wt=javabin&version=2&replicaType=PULL
[junit4] 2> 773027 INFO (qtp901139553-7661) [n:127.0.0.1:35227_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 773027 INFO (qtp1907738058-7690) [n:127.0.0.1:34481_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 773132 INFO (zkCallback-1632-thread-1-processing-n:127.0.0.1:34481_solr) [n:127.0.0.1:34481_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_error_handling_test_cant_connect_to_leader/state.json] for collection [pull_replica_error_handling_test_cant_connect_to_leader] has occurred - updating... (live nodes size: [4])
[junit4] 2> 773132 INFO (zkCallback-1622-thread-1-processing-n:127.0.0.1:35227_solr) [n:127.0.0.1:35227_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_error_handling_test_cant_connect_to_leader/state.json] for collection [pull_replica_error_handling_test_cant_connect_to_leader] has occurred - updating... (live nodes size: [4])
[junit4] 2> 773132 INFO (zkCallback-1632-thread-2-processing-n:127.0.0.1:34481_solr) [n:127.0.0.1:34481_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_error_handling_test_cant_connect_to_leader/state.json] for collection [pull_replica_error_handling_test_cant_connect_to_leader] has occurred - updating... (live nodes size: [4])
[junit4] 2> 773132 INFO (zkCallback-1622-thread-2-processing-n:127.0.0.1:35227_solr) [n:127.0.0.1:35227_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_error_handling_test_cant_connect_to_leader/state.json] for collection [pull_replica_error_handling_test_cant_connect_to_leader] has occurred - updating... (live nodes size: [4])
[junit4] 2> 774037 INFO (qtp1907738058-7690) [n:127.0.0.1:34481_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node4 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 774038 INFO (qtp901139553-7661) [n:127.0.0.1:35227_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node3 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 774043 INFO (qtp1907738058-7690) [n:127.0.0.1:34481_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node4 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2] o.a.s.s.IndexSchema [pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2] Schema name=minimal
[junit4] 2> 774044 INFO (qtp901139553-7661) [n:127.0.0.1:35227_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node3 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_n1] o.a.s.s.IndexSchema [pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_n1] Schema name=minimal
[junit4] 2> 774045 INFO (qtp1907738058-7690) [n:127.0.0.1:34481_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node4 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 774045 INFO (qtp1907738058-7690) [n:127.0.0.1:34481_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node4 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2] o.a.s.c.CoreContainer Creating SolrCore 'pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2' using configuration from collection pull_replica_error_handling_test_cant_connect_to_leader, trusted=true
[junit4] 2> 774046 INFO (qtp1907738058-7690) [n:127.0.0.1:34481_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node4 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34481.solr.core.pull_replica_error_handling_test_cant_connect_to_leader.shard1.replica_p2' (registry 'solr.core.pull_replica_error_handling_test_cant_connect_to_leader.shard1.replica_p2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@336c09cf
[junit4] 2> 774056 INFO (qtp1907738058-7690) [n:127.0.0.1:34481_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node4 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 774056 INFO (qtp1907738058-7690) [n:127.0.0.1:34481_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node4 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2] o.a.s.c.SolrCore [[pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplicaErrorHandling_FFE982745EB68207-001/tempDir-001/node1/pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplicaErrorHandling_FFE982745EB68207-001/tempDir-001/node1/./pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2/data/]
[junit4] 2> 774056 INFO (qtp901139553-7661) [n:127.0.0.1:35227_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node3 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 774056 INFO (qtp901139553-7661) [n:127.0.0.1:35227_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node3 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_n1' using configuration from collection pull_replica_error_handling_test_cant_connect_to_leader, trusted=true
[junit4] 2> 774057 INFO (qtp901139553-7661) [n:127.0.0.1:35227_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node3 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35227.solr.core.pull_replica_error_handling_test_cant_connect_to_leader.shard1.replica_n1' (registry 'solr.core.pull_replica_error_handling_test_cant_connect_to_leader.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@336c09cf
[junit4] 2> 774057 INFO (qtp901139553-7661) [n:127.0.0.1:35227_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node3 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 774057 INFO (qtp901139553-7661) [n:127.0.0.1:35227_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node3 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_n1] o.a.s.c.SolrCore [[pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplicaErrorHandling_FFE982745EB68207-001/tempDir-001/node4/pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplicaErrorHandling_FFE982745EB68207-001/tempDir-001/node4/./pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_n1/data/]
[junit4] 2> 774091 INFO (qtp1907738058-7690) [n:127.0.0.1:34481_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node4 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 774091 INFO (qtp1907738058-7690) [n:127.0.0.1:34481_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node4 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 774092 INFO (qtp1907738058-7690) [n:127.0.0.1:34481_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node4 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2] o.a.s.s.SolrIndexSearcher Opening [Searcher@480fd19c[pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2] main]
[junit4] 2> 774092 INFO (qtp1907738058-7690) [n:127.0.0.1:34481_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node4 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 774093 INFO (qtp1907738058-7690) [n:127.0.0.1:34481_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node4 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 774093 INFO (qtp901139553-7661) [n:127.0.0.1:35227_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node3 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 774093 INFO (qtp901139553-7661) [n:127.0.0.1:35227_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node3 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 774093 INFO (qtp1907738058-7690) [n:127.0.0.1:34481_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node4 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 774093 INFO (qtp1907738058-7690) [n:127.0.0.1:34481_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node4 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2] o.a.s.c.ZkController pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2 starting background replication from leader
[junit4] 2> 774093 INFO (qtp1907738058-7690) [n:127.0.0.1:34481_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node4 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
[junit4] 2> 774094 INFO (searcherExecutor-3503-thread-1-processing-n:127.0.0.1:34481_solr x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2 s:shard1 c:pull_replica_error_handling_test_cant_connect_to_leader r:core_node4) [n:127.0.0.1:34481_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node4 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2] o.a.s.c.SolrCore [pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2] Registered new searcher Searcher@480fd19c[pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 774094 INFO (qtp901139553-7661) [n:127.0.0.1:35227_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node3 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 774094 INFO (qtp901139553-7661) [n:127.0.0.1:35227_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node3 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 774094 INFO (qtp1907738058-7690) [n:127.0.0.1:34481_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node4 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
[junit4] 2> 774094 INFO (qtp1907738058-7690) [n:127.0.0.1:34481_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node4 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_p2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 774094 INFO (qtp901139553-7661) [n:127.0.0.1:35227_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node3 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6705182f[pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_n1] main]
[junit4] 2> 774095 INFO (qtp901139553-7661) [n:127.0.0.1:35227_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node3 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 774095 INFO (qtp901139553-7661) [n:127.0.0.1:35227_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node3 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 774096 INFO (qtp901139553-7661) [n:127.0.0.1:35227_solr c:pull_replica_error_handling_test_cant_connect_to_leader s:shard1 r:core_node3 x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 774096 INFO (searcherExecutor-3504-thread-1-processing-n:127.0.0.1:35227_solr x:pull_replica_error_handling_test_cant_connect_to_leader_shard1_replica_n1 s:shard1 c:pull_replica_error_handling_test
[...truncated too long message...]
adPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 243477 ERROR (qtp919910996-554) [n:127.0.0.1:35623_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.ZkContainer :org.apache.solr.common.SolrException: Error getting leader from zk for shard shard2
[junit4] 2> at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1207)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1079)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1014)
[junit4] 2> at org.apache.solr.core.ZkContainer.lambda$registerInZk$0(ZkContainer.java:181)
[junit4] 2> at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:208)
[junit4] 2> at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:886)
[junit4] 2> at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1047)
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:948)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:735)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:716)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:497)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: org.apache.solr.common.SolrException: No registered leader was found after waiting for 1220000ms , collection: collection1 slice: shard2 saw state=null with live_nodes=[]
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:772)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:737)
[junit4] 2> at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1178)
[junit4] 2> ... 44 more
[junit4] 2>
[junit4] 2> 243488 INFO (qtp919910996-554) [n:127.0.0.1:35623_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection1_shard2_replica_n2&action=CREATE&numShards=2&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=209764
[junit4] 2> 243488 ERROR (SUITE-GraphTest-seed#[8F89C5528A40B32]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 243489 INFO (SUITE-GraphTest-seed#[8F89C5528A40B32]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41789 41789
[junit4] 2> 255233 INFO (Thread-168) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41789 41789
[junit4] 2> 255235 WARN (Thread-168) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/aliases.json
[junit4] 2> 2 /solr/security.json
[junit4] 2> 2 /solr/configs/conf
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/collections/collection1/state.json
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/live_nodes
[junit4] 2> 3 /solr/collections
[junit4] 2>
[junit4] 2> 255235 INFO (SUITE-GraphTest-seed#[8F89C5528A40B32]-worker) [ ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=836, maxMBSortInHeap=7.283536096934936, sim=RandomSimilarity(queryNorm=true): {}, locale=no, timezone=Africa/Bissau
[junit4] 2> NOTE: Linux 4.10.0-40-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=8,threads=1,free=371160024,total=460849152
[junit4] 2> NOTE: All tests run in this JVM: [JsonValidatorTest, SolrExampleEmbeddedTest, TestSolrProperties, HyperbolicCosineEvaluatorTest, QueryResponseTest, TestDelegationTokenResponse, TestLBHttpSolrClient, UniformDistributionEvaluatorTest, SubtractEvaluatorTest, ModuloEvaluatorTest, JDBCStreamTest, RoundEvaluatorTest, SolrZkClientTest, GraphTest]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=GraphTest -Dtests.seed=8F89C5528A40B32 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=no -Dtests.timezone=Africa/Bissau -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J1 | GraphTest (suite) <<<
[junit4] > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:36255/solr: create the collection time out:180s
[junit4] > at __randomizedtesting.SeedInfo.seed([8F89C5528A40B32]:0)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1103)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:883)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
[junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
[junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
[junit4] > at org.apache.solr.client.solrj.io.graph.GraphTest.setupCluster(GraphTest.java:62)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] Completed [94/146 (1!)] on J1 in 222.19s, 0 tests, 1 error <<< FAILURES!
[...truncated 46588 lines...]
[JENKINS] Lucene-Solr-7.x-Linux (32bit/jdk1.8.0_144) - Build # 958
- Failure!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/958/
Java: 32bit/jdk1.8.0_144 -server -XX:+UseG1GC
1 tests failed.
FAILED: org.apache.solr.core.snapshots.TestSolrCloudSnapshots.testSnapshots
Error Message:
Error from server at http://127.0.0.1:42827/solr: create the collection time out:180s
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:42827/solr: create the collection time out:180s
at __randomizedtesting.SeedInfo.seed([B4276B5D926F10BF:826BEFC2E73612F0]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1103)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:883)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.core.snapshots.TestSolrCloudSnapshots.testSnapshots(TestSolrCloudSnapshots.java:86)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13907 lines...]
[junit4] Suite: org.apache.solr.core.snapshots.TestSolrCloudSnapshots
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.snapshots.TestSolrCloudSnapshots_B4276B5D926F10BF-001/init-core-data-001
[junit4] 2> 2432386 WARN (SUITE-TestSolrCloudSnapshots-seed#[B4276B5D926F10BF]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=56 numCloses=56
[junit4] 2> 2432386 INFO (SUITE-TestSolrCloudSnapshots-seed#[B4276B5D926F10BF]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 2432387 INFO (SUITE-TestSolrCloudSnapshots-seed#[B4276B5D926F10BF]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
[junit4] 2> 2432387 INFO (SUITE-TestSolrCloudSnapshots-seed#[B4276B5D926F10BF]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 2432388 INFO (SUITE-TestSolrCloudSnapshots-seed#[B4276B5D926F10BF]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.snapshots.TestSolrCloudSnapshots_B4276B5D926F10BF-001/tempDir-001
[junit4] 2> 2432388 INFO (SUITE-TestSolrCloudSnapshots-seed#[B4276B5D926F10BF]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2432388 INFO (Thread-10083) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2432388 INFO (Thread-10083) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2432390 ERROR (Thread-10083) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2432488 INFO (SUITE-TestSolrCloudSnapshots-seed#[B4276B5D926F10BF]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:46189
[junit4] 2> 2432491 INFO (zkConnectionManagerCallback-4690-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2432496 INFO (jetty-launcher-4687-thread-2) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2432496 INFO (jetty-launcher-4687-thread-1) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2432497 INFO (jetty-launcher-4687-thread-3) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2432498 INFO (jetty-launcher-4687-thread-4) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2432499 INFO (jetty-launcher-4687-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5b93e{/solr,null,AVAILABLE}
[junit4] 2> 2432500 INFO (jetty-launcher-4687-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1f69911{/solr,null,AVAILABLE}
[junit4] 2> 2432500 INFO (jetty-launcher-4687-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1ab63ef{/solr,null,AVAILABLE}
[junit4] 2> 2432500 INFO (jetty-launcher-4687-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@984204{HTTP/1.1,[http/1.1]}{127.0.0.1:40435}
[junit4] 2> 2432500 INFO (jetty-launcher-4687-thread-3) [ ] o.e.j.s.Server Started @2434075ms
[junit4] 2> 2432500 INFO (jetty-launcher-4687-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40435}
[junit4] 2> 2432501 INFO (jetty-launcher-4687-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1540b22{/solr,null,AVAILABLE}
[junit4] 2> 2432501 INFO (jetty-launcher-4687-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@67eb3f{HTTP/1.1,[http/1.1]}{127.0.0.1:46589}
[junit4] 2> 2432501 INFO (jetty-launcher-4687-thread-4) [ ] o.e.j.s.Server Started @2434076ms
[junit4] 2> 2432501 INFO (jetty-launcher-4687-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@179532c{HTTP/1.1,[http/1.1]}{127.0.0.1:40149}
[junit4] 2> 2432502 ERROR (jetty-launcher-4687-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2432502 INFO (jetty-launcher-4687-thread-2) [ ] o.e.j.s.Server Started @2434077ms
[junit4] 2> 2432502 INFO (jetty-launcher-4687-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 2432502 INFO (jetty-launcher-4687-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40149}
[junit4] 2> 2432501 INFO (jetty-launcher-4687-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46589}
[junit4] 2> 2432502 INFO (jetty-launcher-4687-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2432503 ERROR (jetty-launcher-4687-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2432503 ERROR (jetty-launcher-4687-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2432503 INFO (jetty-launcher-4687-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 2432503 INFO (jetty-launcher-4687-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 2432503 INFO (jetty-launcher-4687-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2432503 INFO (jetty-launcher-4687-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2432503 INFO (jetty-launcher-4687-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2432503 INFO (jetty-launcher-4687-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2432503 INFO (jetty-launcher-4687-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-08T11:54:58.742Z
[junit4] 2> 2432503 INFO (jetty-launcher-4687-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-08T11:54:58.742Z
[junit4] 2> 2432502 INFO (jetty-launcher-4687-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@127e5e0{HTTP/1.1,[http/1.1]}{127.0.0.1:42827}
[junit4] 2> 2432504 INFO (jetty-launcher-4687-thread-1) [ ] o.e.j.s.Server Started @2434079ms
[junit4] 2> 2432503 INFO (jetty-launcher-4687-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2432504 INFO (jetty-launcher-4687-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-08T11:54:58.743Z
[junit4] 2> 2432504 INFO (jetty-launcher-4687-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42827}
[junit4] 2> 2432504 ERROR (jetty-launcher-4687-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2432504 INFO (jetty-launcher-4687-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 2432504 INFO (jetty-launcher-4687-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2432504 INFO (jetty-launcher-4687-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2432504 INFO (jetty-launcher-4687-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-12-08T11:54:58.743Z
[junit4] 2> 2432506 INFO (zkConnectionManagerCallback-4692-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2432506 INFO (zkConnectionManagerCallback-4694-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2432506 INFO (zkConnectionManagerCallback-4696-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2432507 INFO (zkConnectionManagerCallback-4698-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2432507 INFO (jetty-launcher-4687-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2432507 INFO (jetty-launcher-4687-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2432507 INFO (jetty-launcher-4687-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2432509 INFO (jetty-launcher-4687-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2432521 INFO (jetty-launcher-4687-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46189/solr
[junit4] 2> 2432522 INFO (jetty-launcher-4687-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46189/solr
[junit4] 2> 2432523 INFO (zkConnectionManagerCallback-4705-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2432524 INFO (jetty-launcher-4687-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46189/solr
[junit4] 2> 2432524 INFO (jetty-launcher-4687-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46189/solr
[junit4] 2> 2432526 INFO (zkConnectionManagerCallback-4709-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2432527 INFO (zkConnectionManagerCallback-4711-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2432527 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x16035fa84c40005, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2432527 INFO (zkConnectionManagerCallback-4714-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2432530 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x16035fa84c40006, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2432531 INFO (zkConnectionManagerCallback-4716-thread-1-processing-n:127.0.0.1:46589_solr) [n:127.0.0.1:46589_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2432532 INFO (zkConnectionManagerCallback-4718-thread-1-processing-n:127.0.0.1:42827_solr) [n:127.0.0.1:42827_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2432532 INFO (zkConnectionManagerCallback-4720-thread-1-processing-n:127.0.0.1:40149_solr) [n:127.0.0.1:40149_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2432536 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x16035fa84c40007, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2432541 INFO (zkConnectionManagerCallback-4722-thread-1-processing-n:127.0.0.1:40435_solr) [n:127.0.0.1:40435_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2432611 INFO (jetty-launcher-4687-thread-3) [n:127.0.0.1:40435_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2432612 INFO (jetty-launcher-4687-thread-3) [n:127.0.0.1:40435_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40435_solr
[junit4] 2> 2432617 INFO (jetty-launcher-4687-thread-3) [n:127.0.0.1:40435_solr ] o.a.s.c.Overseer Overseer (id=99138541887684620-127.0.0.1:40435_solr-n_0000000000) starting
[junit4] 2> 2432618 INFO (jetty-launcher-4687-thread-2) [n:127.0.0.1:40149_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2432618 INFO (jetty-launcher-4687-thread-4) [n:127.0.0.1:46589_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2432618 INFO (jetty-launcher-4687-thread-1) [n:127.0.0.1:42827_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2432619 INFO (jetty-launcher-4687-thread-2) [n:127.0.0.1:40149_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40149_solr
[junit4] 2> 2432619 INFO (jetty-launcher-4687-thread-4) [n:127.0.0.1:46589_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46589_solr
[junit4] 2> 2432620 INFO (jetty-launcher-4687-thread-1) [n:127.0.0.1:42827_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42827_solr
[junit4] 2> 2432634 INFO (zkCallback-4717-thread-1-processing-n:127.0.0.1:42827_solr) [n:127.0.0.1:42827_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2432634 INFO (zkCallback-4719-thread-1-processing-n:127.0.0.1:40149_solr) [n:127.0.0.1:40149_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2432634 INFO (zkCallback-4715-thread-1-processing-n:127.0.0.1:46589_solr) [n:127.0.0.1:46589_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2432634 INFO (zkCallback-4721-thread-1-processing-n:127.0.0.1:40435_solr) [n:127.0.0.1:40435_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2432656 INFO (jetty-launcher-4687-thread-3) [n:127.0.0.1:40435_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40435_solr
[junit4] 2> 2432657 INFO (zkCallback-4717-thread-1-processing-n:127.0.0.1:42827_solr) [n:127.0.0.1:42827_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2432657 INFO (zkCallback-4715-thread-1-processing-n:127.0.0.1:46589_solr) [n:127.0.0.1:46589_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2432657 INFO (zkCallback-4721-thread-1-processing-n:127.0.0.1:40435_solr) [n:127.0.0.1:40435_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2432678 INFO (zkCallback-4719-thread-1-processing-n:127.0.0.1:40149_solr) [n:127.0.0.1:40149_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2432718 INFO (jetty-launcher-4687-thread-4) [n:127.0.0.1:46589_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46589.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@928f3b
[junit4] 2> 2432730 INFO (jetty-launcher-4687-thread-4) [n:127.0.0.1:46589_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46589.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@928f3b
[junit4] 2> 2432730 INFO (jetty-launcher-4687-thread-4) [n:127.0.0.1:46589_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46589.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@928f3b
[junit4] 2> 2432731 INFO (jetty-launcher-4687-thread-4) [n:127.0.0.1:46589_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.snapshots.TestSolrCloudSnapshots_B4276B5D926F10BF-001/tempDir-001/node4/.
[junit4] 2> 2432735 INFO (jetty-launcher-4687-thread-3) [n:127.0.0.1:40435_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40435.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@928f3b
[junit4] 2> 2432744 INFO (jetty-launcher-4687-thread-3) [n:127.0.0.1:40435_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40435.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@928f3b
[junit4] 2> 2432744 INFO (jetty-launcher-4687-thread-3) [n:127.0.0.1:40435_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40435.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@928f3b
[junit4] 2> 2432746 INFO (jetty-launcher-4687-thread-3) [n:127.0.0.1:40435_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.snapshots.TestSolrCloudSnapshots_B4276B5D926F10BF-001/tempDir-001/node3/.
[junit4] 2> 2432760 INFO (jetty-launcher-4687-thread-2) [n:127.0.0.1:40149_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40149.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@928f3b
[junit4] 2> 2432768 INFO (jetty-launcher-4687-thread-2) [n:127.0.0.1:40149_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40149.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@928f3b
[junit4] 2> 2432768 INFO (jetty-launcher-4687-thread-2) [n:127.0.0.1:40149_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40149.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@928f3b
[junit4] 2> 2432769 INFO (jetty-launcher-4687-thread-1) [n:127.0.0.1:42827_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42827.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@928f3b
[junit4] 2> 2432769 INFO (jetty-launcher-4687-thread-2) [n:127.0.0.1:40149_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.snapshots.TestSolrCloudSnapshots_B4276B5D926F10BF-001/tempDir-001/node2/.
[junit4] 2> 2432777 INFO (jetty-launcher-4687-thread-1) [n:127.0.0.1:42827_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42827.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@928f3b
[junit4] 2> 2432777 INFO (jetty-launcher-4687-thread-1) [n:127.0.0.1:42827_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42827.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@928f3b
[junit4] 2> 2432779 INFO (jetty-launcher-4687-thread-1) [n:127.0.0.1:42827_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.snapshots.TestSolrCloudSnapshots_B4276B5D926F10BF-001/tempDir-001/node1/.
[junit4] 2> 2432842 INFO (zkConnectionManagerCallback-4729-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2432844 INFO (zkConnectionManagerCallback-4733-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2432845 INFO (SUITE-TestSolrCloudSnapshots-seed#[B4276B5D926F10BF]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2432846 INFO (SUITE-TestSolrCloudSnapshots-seed#[B4276B5D926F10BF]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46189/solr ready
[junit4] 2> 2432869 INFO (TEST-TestSolrCloudSnapshots.testSnapshots-seed#[B4276B5D926F10BF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSnapshots
[junit4] 2> 2432870 INFO (qtp28825050-26287) [n:127.0.0.1:42827_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf1&name=SolrCloudSnapshots&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2432872 INFO (OverseerThreadFactory-7546-thread-1-processing-n:127.0.0.1:40435_solr) [n:127.0.0.1:40435_solr ] o.a.s.c.CreateCollectionCmd Create collection SolrCloudSnapshots
[junit4] 2> 2432975 INFO (OverseerStateUpdate-99138541887684620-127.0.0.1:40435_solr-n_0000000000) [n:127.0.0.1:40435_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"SolrCloudSnapshots",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"SolrCloudSnapshots_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:46589/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2432976 INFO (OverseerStateUpdate-99138541887684620-127.0.0.1:40435_solr-n_0000000000) [n:127.0.0.1:40435_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"SolrCloudSnapshots",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"SolrCloudSnapshots_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:42827/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2432977 INFO (OverseerStateUpdate-99138541887684620-127.0.0.1:40435_solr-n_0000000000) [n:127.0.0.1:40435_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"SolrCloudSnapshots",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"SolrCloudSnapshots_shard2_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:40435/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2432977 INFO (OverseerStateUpdate-99138541887684620-127.0.0.1:40435_solr-n_0000000000) [n:127.0.0.1:40435_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"SolrCloudSnapshots",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"SolrCloudSnapshots_shard2_replica_n6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:40149/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2433179 INFO (qtp28825050-26294) [n:127.0.0.1:42827_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf1&newCollection=true&name=SolrCloudSnapshots_shard1_replica_n2&action=CREATE&numShards=2&collection=SolrCloudSnapshots&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2433179 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=SolrCloudSnapshots_shard1_replica_n1&action=CREATE&numShards=2&collection=SolrCloudSnapshots&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2433179 INFO (qtp28825050-26294) [n:127.0.0.1:42827_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2433179 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2433179 INFO (qtp21896525-26279) [n:127.0.0.1:40149_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf1&newCollection=true&name=SolrCloudSnapshots_shard2_replica_n6&action=CREATE&numShards=2&collection=SolrCloudSnapshots&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2433179 INFO (qtp21827506-26271) [n:127.0.0.1:40435_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=SolrCloudSnapshots_shard2_replica_n4&action=CREATE&numShards=2&collection=SolrCloudSnapshots&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2433179 INFO (qtp21896525-26279) [n:127.0.0.1:40149_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2433179 INFO (qtp21827506-26271) [n:127.0.0.1:40435_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2433283 INFO (zkCallback-4717-thread-1-processing-n:127.0.0.1:42827_solr) [n:127.0.0.1:42827_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/SolrCloudSnapshots/state.json] for collection [SolrCloudSnapshots] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2433283 INFO (zkCallback-4721-thread-1-processing-n:127.0.0.1:40435_solr) [n:127.0.0.1:40435_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/SolrCloudSnapshots/state.json] for collection [SolrCloudSnapshots] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2433283 INFO (zkCallback-4719-thread-1-processing-n:127.0.0.1:40149_solr) [n:127.0.0.1:40149_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/SolrCloudSnapshots/state.json] for collection [SolrCloudSnapshots] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2433283 INFO (zkCallback-4715-thread-2-processing-n:127.0.0.1:46589_solr) [n:127.0.0.1:46589_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/SolrCloudSnapshots/state.json] for collection [SolrCloudSnapshots] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2433283 INFO (zkCallback-4721-thread-2-processing-n:127.0.0.1:40435_solr) [n:127.0.0.1:40435_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/SolrCloudSnapshots/state.json] for collection [SolrCloudSnapshots] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2433283 INFO (zkCallback-4715-thread-1-processing-n:127.0.0.1:46589_solr) [n:127.0.0.1:46589_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/SolrCloudSnapshots/state.json] for collection [SolrCloudSnapshots] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2433283 INFO (zkCallback-4717-thread-2-processing-n:127.0.0.1:42827_solr) [n:127.0.0.1:42827_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/SolrCloudSnapshots/state.json] for collection [SolrCloudSnapshots] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2433283 INFO (zkCallback-4719-thread-2-processing-n:127.0.0.1:40149_solr) [n:127.0.0.1:40149_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/SolrCloudSnapshots/state.json] for collection [SolrCloudSnapshots] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2434193 INFO (qtp28825050-26294) [n:127.0.0.1:42827_solr c:SolrCloudSnapshots s:shard1 r:core_node5 x:SolrCloudSnapshots_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 2434193 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr c:SolrCloudSnapshots s:shard1 r:core_node3 x:SolrCloudSnapshots_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 2434193 INFO (qtp21896525-26279) [n:127.0.0.1:40149_solr c:SolrCloudSnapshots s:shard2 r:core_node8 x:SolrCloudSnapshots_shard2_replica_n6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 2434193 INFO (qtp21827506-26271) [n:127.0.0.1:40435_solr c:SolrCloudSnapshots s:shard2 r:core_node7 x:SolrCloudSnapshots_shard2_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 2434212 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr c:SolrCloudSnapshots s:shard1 r:core_node3 x:SolrCloudSnapshots_shard1_replica_n1] o.a.s.s.IndexSchema [SolrCloudSnapshots_shard1_replica_n1] Schema name=minimal
[junit4] 2> 2434212 INFO (qtp28825050-26294) [n:127.0.0.1:42827_solr c:SolrCloudSnapshots s:shard1 r:core_node5 x:SolrCloudSnapshots_shard1_replica_n2] o.a.s.s.IndexSchema [SolrCloudSnapshots_shard1_replica_n2] Schema name=minimal
[junit4] 2> 2434214 INFO (qtp21827506-26271) [n:127.0.0.1:40435_solr c:SolrCloudSnapshots s:shard2 r:core_node7 x:SolrCloudSnapshots_shard2_replica_n4] o.a.s.s.IndexSchema [SolrCloudSnapshots_shard2_replica_n4] Schema name=minimal
[junit4] 2> 2434214 INFO (qtp21896525-26279) [n:127.0.0.1:40149_solr c:SolrCloudSnapshots s:shard2 r:core_node8 x:SolrCloudSnapshots_shard2_replica_n6] o.a.s.s.IndexSchema [SolrCloudSnapshots_shard2_replica_n6] Schema name=minimal
[junit4] 2> 2434215 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr c:SolrCloudSnapshots s:shard1 r:core_node3 x:SolrCloudSnapshots_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2434215 INFO (qtp28825050-26294) [n:127.0.0.1:42827_solr c:SolrCloudSnapshots s:shard1 r:core_node5 x:SolrCloudSnapshots_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2434215 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr c:SolrCloudSnapshots s:shard1 r:core_node3 x:SolrCloudSnapshots_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'SolrCloudSnapshots_shard1_replica_n1' using configuration from collection SolrCloudSnapshots, trusted=true
[junit4] 2> 2434215 INFO (qtp28825050-26294) [n:127.0.0.1:42827_solr c:SolrCloudSnapshots s:shard1 r:core_node5 x:SolrCloudSnapshots_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'SolrCloudSnapshots_shard1_replica_n2' using configuration from collection SolrCloudSnapshots, trusted=true
[junit4] 2> 2434216 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr c:SolrCloudSnapshots s:shard1 r:core_node3 x:SolrCloudSnapshots_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46589.solr.core.SolrCloudSnapshots.shard1.replica_n1' (registry 'solr.core.SolrCloudSnapshots.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@928f3b
[junit4] 2> 2434216 INFO (qtp28825050-26294) [n:127.0.0.1:42827_solr c:SolrCloudSnapshots s:shard1 r:core_node5 x:SolrCloudSnapshots_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42827.solr.core.SolrCloudSnapshots.shard1.replica_n2' (registry 'solr.core.SolrCloudSnapshots.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@928f3b
[junit4] 2> 2434216 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr c:SolrCloudSnapshots s:shard1 r:core_node3 x:SolrCloudSnapshots_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2434216 INFO (qtp28825050-26294) [n:127.0.0.1:42827_solr c:SolrCloudSnapshots s:shard1 r:core_node5 x:SolrCloudSnapshots_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2434216 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr c:SolrCloudSnapshots s:shard1 r:core_node3 x:SolrCloudSnapshots_shard1_replica_n1] o.a.s.c.SolrCore [[SolrCloudSnapshots_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.snapshots.TestSolrCloudSnapshots_B4276B5D926F10BF-001/tempDir-001/node4/SolrCloudSnapshots_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.snapshots.TestSolrCloudSnapshots_B4276B5D926F10BF-001/tempDir-001/node4/./SolrCloudSnapshots_shard1_replica_n1/data/]
[junit4] 2> 2434216 INFO (qtp28825050-26294) [n:127.0.0.1:42827_solr c:SolrCloudSnapshots s:shard1 r:core_node5 x:SolrCloudSnapshots_shard1_replica_n2] o.a.s.c.SolrCore [[SolrCloudSnapshots_shard1_replica_n2] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.snapshots.TestSolrCloudSnapshots_B4276B5D926F10BF-001/tempDir-001/node1/SolrCloudSnapshots_shard1_replica_n2], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.snapshots.TestSolrCloudSnapshots_B4276B5D926F10BF-001/tempDir-001/node1/./SolrCloudSnapshots_shard1_replica_n2/data/]
[junit4] 2> 2434217 INFO (qtp21896525-26279) [n:127.0.0.1:40149_solr c:SolrCloudSnapshots s:shard2 r:core_node8 x:SolrCloudSnapshots_shard2_replica_n6] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2434217 INFO (qtp21827506-26271) [n:127.0.0.1:40435_solr c:SolrCloudSnapshots s:shard2 r:core_node7 x:SolrCloudSnapshots_shard2_replica_n4] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2434217 INFO (qtp21896525-26279) [n:127.0.0.1:40149_solr c:SolrCloudSnapshots s:shard2 r:core_node8 x:SolrCloudSnapshots_shard2_replica_n6] o.a.s.c.CoreContainer Creating SolrCore 'SolrCloudSnapshots_shard2_replica_n6' using configuration from collection SolrCloudSnapshots, trusted=true
[junit4] 2> 2434217 INFO (qtp21827506-26271) [n:127.0.0.1:40435_solr c:SolrCloudSnapshots s:shard2 r:core_node7 x:SolrCloudSnapshots_shard2_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'SolrCloudSnapshots_shard2_replica_n4' using configuration from collection SolrCloudSnapshots, trusted=true
[junit4] 2> 2434218 INFO (qtp21896525-26279) [n:127.0.0.1:40149_solr c:SolrCloudSnapshots s:shard2 r:core_node8 x:SolrCloudSnapshots_shard2_replica_n6] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40149.solr.core.SolrCloudSnapshots.shard2.replica_n6' (registry 'solr.core.SolrCloudSnapshots.shard2.replica_n6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@928f3b
[junit4] 2> 2434218 INFO (qtp21827506-26271) [n:127.0.0.1:40435_solr c:SolrCloudSnapshots s:shard2 r:core_node7 x:SolrCloudSnapshots_shard2_replica_n4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40435.solr.core.SolrCloudSnapshots.shard2.replica_n4' (registry 'solr.core.SolrCloudSnapshots.shard2.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@928f3b
[junit4] 2> 2434218 INFO (qtp21896525-26279) [n:127.0.0.1:40149_solr c:SolrCloudSnapshots s:shard2 r:core_node8 x:SolrCloudSnapshots_shard2_replica_n6] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2434218 INFO (qtp21827506-26271) [n:127.0.0.1:40435_solr c:SolrCloudSnapshots s:shard2 r:core_node7 x:SolrCloudSnapshots_shard2_replica_n4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2434219 INFO (qtp21896525-26279) [n:127.0.0.1:40149_solr c:SolrCloudSnapshots s:shard2 r:core_node8 x:SolrCloudSnapshots_shard2_replica_n6] o.a.s.c.SolrCore [[SolrCloudSnapshots_shard2_replica_n6] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.snapshots.TestSolrCloudSnapshots_B4276B5D926F10BF-001/tempDir-001/node2/SolrCloudSnapshots_shard2_replica_n6], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.snapshots.TestSolrCloudSnapshots_B4276B5D926F10BF-001/tempDir-001/node2/./SolrCloudSnapshots_shard2_replica_n6/data/]
[junit4] 2> 2434219 INFO (qtp21827506-26271) [n:127.0.0.1:40435_solr c:SolrCloudSnapshots s:shard2 r:core_node7 x:SolrCloudSnapshots_shard2_replica_n4] o.a.s.c.SolrCore [[SolrCloudSnapshots_shard2_replica_n4] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.snapshots.TestSolrCloudSnapshots_B4276B5D926F10BF-001/tempDir-001/node3/SolrCloudSnapshots_shard2_replica_n4], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.snapshots.TestSolrCloudSnapshots_B4276B5D926F10BF-001/tempDir-001/node3/./SolrCloudSnapshots_shard2_replica_n4/data/]
[junit4] 2> 2434436 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr c:SolrCloudSnapshots s:shard1 r:core_node3 x:SolrCloudSnapshots_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2434436 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr c:SolrCloudSnapshots s:shard1 r:core_node3 x:SolrCloudSnapshots_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2434437 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr c:SolrCloudSnapshots s:shard1 r:core_node3 x:SolrCloudSnapshots_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2434437 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr c:SolrCloudSnapshots s:shard1 r:core_node3 x:SolrCloudSnapshots_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2434438 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr c:SolrCloudSnapshots s:shard1 r:core_node3 x:SolrCloudSnapshots_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@967e0d[SolrCloudSnapshots_shard1_replica_n1] main]
[junit4] 2> 2434439 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr c:SolrCloudSnapshots s:shard1 r:core_node3 x:SolrCloudSnapshots_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2434439 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr c:SolrCloudSnapshots s:shard1 r:core_node3 x:SolrCloudSnapshots_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2434440 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr c:SolrCloudSnapshots s:shard1 r:core_node3 x:SolrCloudSnapshots_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2434441 INFO (searcherExecutor-7555-thread-1-processing-n:127.0.0.1:46589_solr x:SolrCloudSnapshots_shard1_replica_n1 s:shard1 c:SolrCloudSnapshots r:core_node3) [n:127.0.0.1:46589_solr c:SolrCloudSnapshots s:shard1 r:core_node3 x:SolrCloudSnapshots_shard1_replica_n1] o.a.s.c.SolrCore [SolrCloudSnapshots_shard1_replica_n1] Registered new searcher Searcher@967e0d[SolrCloudSnapshots_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2434441 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr c:SolrCloudSnapshots s:shard1 r:core_node3 x:SolrCloudSnapshots_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1586216672354631680
[junit4] 2> 2434444 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr c:SolrCloudSnapshots s:shard1 r:core_node3 x:SolrCloudSnapshots_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
[junit4] 2> 2434444 INFO (qtp28825050-26294) [n:127.0.0.1:42827_solr c:SolrCloudSnapshots s:shard1 r:core_node5 x:SolrCloudSnapshots_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2434444 INFO (qtp28825050-26294) [n:127.0.0.1:42827_solr c:SolrCloudSnapshots s:shard1 r:core_node5 x:SolrCloudSnapshots_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2434445 INFO (qtp28825050-26294) [n:127.0.0.1:42827_solr c:SolrCloudSnapshots s:shard1 r:core_node5 x:SolrCloudSnapshots_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2434445 INFO (qtp28825050-26294) [n:127.0.0.1:42827_solr c:SolrCloudSnapshots s:shard1 r:core_node5 x:SolrCloudSnapshots_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2434447 INFO (qtp28825050-26294) [n:127.0.0.1:42827_solr c:SolrCloudSnapshots s:shard1 r:core_node5 x:SolrCloudSnapshots_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@fa46c1[SolrCloudSnapshots_shard1_replica_n2] main]
[junit4] 2> 2434447 INFO (qtp28825050-26294) [n:127.0.0.1:42827_solr c:SolrCloudSnapshots s:shard1 r:core_node5 x:SolrCloudSnapshots_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2434448 INFO (qtp28825050-26294) [n:127.0.0.1:42827_solr c:SolrCloudSnapshots s:shard1 r:core_node5 x:SolrCloudSnapshots_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2434448 INFO (qtp28825050-26294) [n:127.0.0.1:42827_solr c:SolrCloudSnapshots s:shard1 r:core_node5 x:SolrCloudSnapshots_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2434449 INFO (searcherExecutor-7556-thread-1-processing-n:127.0.0.1:42827_solr x:SolrCloudSnapshots_shard1_replica_n2 s:shard1 c:SolrCloudSnapshots r:core_node5) [n:127.0.0.1:42827_solr c:SolrCloudSnapshots s:shard1 r:core_node5 x:SolrCloudSnapshots_shard1_replica_n2] o.a.s.c.SolrCore [SolrCloudSnapshots_shard1_replica_n2] Registered new searcher Searcher@fa46c1[SolrCloudSnapshots_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2434449 INFO (qtp28825050-26294) [n:127.0.0.1:42827_solr c:SolrCloudSnapshots s:shard1 r:core_node5 x:SolrCloudSnapshots_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1586216672363020288
[junit4] 2> 2434455 INFO (qtp21896525-26279) [n:127.0.0.1:40149_solr c:SolrCloudSnapshots s:shard2 r:core_node8 x:SolrCloudSnapshots_shard2_replica_n6] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2434455 INFO (qtp21827506-26271) [n:127.0.0.1:40435_solr c:SolrCloudSnapshots s:shard2 r:core_node7 x:SolrCloudSnapshots_shard2_replica_n4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2434455 INFO (qtp21896525-26279) [n:127.0.0.1:40149_solr c:SolrCloudSnapshots s:shard2 r:core_node8 x:SolrCloudSnapshots_shard2_replica_n6] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2434455 INFO (qtp21827506-26271) [n:127.0.0.1:40435_solr c:SolrCloudSnapshots s:shard2 r:core_node7 x:SolrCloudSnapshots_shard2_replica_n4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2434456 INFO (qtp21827506-26271) [n:127.0.0.1:40435_solr c:SolrCloudSnapshots s:shard2 r:core_node7 x:SolrCloudSnapshots_shard2_replica_n4] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2434456 INFO (qtp21896525-26279) [n:127.0.0.1:40149_solr c:SolrCloudSnapshots s:shard2 r:core_node8 x:SolrCloudSnapshots_shard2_replica_n6] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2434456 INFO (qtp21827506-26271) [n:127.0.0.1:40435_solr c:SolrCloudSnapshots s:shard2 r:core_node7 x:SolrCloudSnapshots_shard2_replica_n4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2434456 INFO (qtp21896525-26279) [n:127.0.0.1:40149_solr c:SolrCloudSnapshots s:shard2 r:core_node8 x:SolrCloudSnapshots_shard2_replica_n6] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2434458 INFO (qtp21827506-26271) [n:127.0.0.1:40435_solr c:SolrCloudSnapshots s:shard2 r:core_node7 x:SolrCloudSnapshots_shard2_replica_n4] o.a.s.s.SolrIndexSearcher Opening [Searcher@4b6598[SolrCloudSnapshots_shard2_replica_n4] main]
[junit4] 2> 2434458 INFO (qtp21896525-26279) [n:127.0.0.1:40149_solr c:SolrCloudSnapshots s:shard2 r:core_node8 x:SolrCloudSnapshots_shard2_replica_n6] o.a.s.s.SolrIndexSearcher Opening [Searcher@186f7a5[SolrCloudSnapshots_shard2_replica_n6] main]
[junit4] 2> 2434459 INFO (qtp21827506-26271) [n:127.0.0.1:40435_solr c:SolrCloudSnapshots s:shard2 r:core_node7 x:SolrCloudSnapshots_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2434459 INFO (qtp21896525-26279) [n:127.0.0.1:40149_solr c:SolrCloudSnapshots s:shard2 r:core_node8 x:SolrCloudSnapshots_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2434459 INFO (qtp21827506-26271) [n:127.0.0.1:40435_solr c:SolrCloudSnapshots s:shard2 r:core_node7 x:SolrCloudSnapshots_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2434459 INFO (qtp21896525-26279) [n:127.0.0.1:40149_solr c:SolrCloudSnapshots s:shard2 r:core_node8 x:SolrCloudSnapshots_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2434460 INFO (qtp21827506-26271) [n:127.0.0.1:40435_solr c:SolrCloudSnapshots s:shard2 r:core_node7 x:SolrCloudSnapshots_shard2_replica_n4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2434460 INFO (qtp21896525-26279) [n:127.0.0.1:40149_solr c:SolrCloudSnapshots s:shard2 r:core_node8 x:SolrCloudSnapshots_shard2_replica_n6] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2434460 INFO (searcherExecutor-7558-thread-1-processing-n:127.0.0.1:40435_solr x:SolrCloudSnapshots_shard2_replica_n4 s:shard2 c:SolrCloudSnapshots r:core_node7) [n:127.0.0.1:40435_solr c:SolrCloudSnapshots s:shard2 r:core_node7 x:SolrCloudSnapshots_shard2_replica_n4] o.a.s.c.SolrCore [SolrCloudSnapshots_shard2_replica_n4] Registered new searcher Searcher@4b6598[SolrCloudSnapshots_shard2_replica_n4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2434460 INFO (searcherExecutor-7557-thread-1-processing-n:127.0.0.1:40149_solr x:SolrCloudSnapshots_shard2_replica_n6 s:shard2 c:SolrCloudSnapshots r:core_node8) [n:127.0.0.1:40149_solr c:SolrCloudSnapshots s:shard2 r:core_node8 x:SolrCloudSnapshots_shard2_replica_n6] o.a.s.c.SolrCore [SolrCloudSnapshots_shard2_replica_n6] Registered new searcher Searcher@186f7a5[SolrCloudSnapshots_shard2_replica_n6] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2434460 INFO (qtp21827506-26271) [n:127.0.0.1:40435_solr c:SolrCloudSnapshots s:shard2 r:core_node7 x:SolrCloudSnapshots_shard2_replica_n4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1586216672374554624
[junit4] 2> 2434460 INFO (qtp21896525-26279) [n:127.0.0.1:40149_solr c:SolrCloudSnapshots s:shard2 r:core_node8 x:SolrCloudSnapshots_shard2_replica_n6] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1586216672374554624
[junit4] 2> 2434463 INFO (qtp21896525-26279) [n:127.0.0.1:40149_solr c:SolrCloudSnapshots s:shard2 r:core_node8 x:SolrCloudSnapshots_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2434464 INFO (qtp21896525-26279) [n:127.0.0.1:40149_solr c:SolrCloudSnapshots s:shard2 r:core_node8 x:SolrCloudSnapshots_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2434464 INFO (qtp21896525-26279) [n:127.0.0.1:40149_solr c:SolrCloudSnapshots s:shard2 r:core_node8 x:SolrCloudSnapshots_shard2_replica_n6] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40149/solr/SolrCloudSnapshots_shard2_replica_n6/
[junit4] 2> 2434464 INFO (qtp21896525-26279) [n:127.0.0.1:40149_solr c:SolrCloudSnapshots s:shard2 r:core_node8 x:SolrCloudSnapshots_shard2_replica_n6] o.a.s.u.PeerSync PeerSync: core=SolrCloudSnapshots_shard2_replica_n6 url=http://127.0.0.1:40149/solr START replicas=[http://127.0.0.1:40435/solr/SolrCloudSnapshots_shard2_replica_n4/] nUpdates=100
[junit4] 2> 2434465 INFO (qtp21827506-26278) [n:127.0.0.1:40435_solr c:SolrCloudSnapshots s:shard2 r:core_node7 x:SolrCloudSnapshots_shard2_replica_n4] o.a.s.c.S.Request [SolrCloudSnapshots_shard2_replica_n4] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2434764 INFO (qtp21896525-26279) [n:127.0.0.1:40149_solr c:SolrCloudSnapshots s:shard2 r:core_node8 x:SolrCloudSnapshots_shard2_replica_n6] o.a.s.u.PeerSync PeerSync: core=SolrCloudSnapshots_shard2_replica_n6 url=http://127.0.0.1:40149/solr DONE. We have no versions. sync failed.
[junit4] 2> 2434764 INFO (qtp21896525-26279) [n:127.0.0.1:40149_solr c:SolrCloudSnapshots s:shard2 r:core_node8 x:SolrCloudSnapshots_shard2_replica_n6] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 2434764 INFO (qtp21896525-26279) [n:127.0.0.1:40149_solr c:SolrCloudSnapshots s:shard2 r:core_node8 x:SolrCloudSnapshots_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 2434764 INFO (qtp21896525-26279) [n:127.0.0.1:40149_solr c:SolrCloudSnapshots s:shard2 r:core_node8 x:SolrCloudSnapshots_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2434766 INFO (qtp21896525-26279) [n:127.0.0.1:40149_solr c:SolrCloudSnapshots s:shard2 r:core_node8 x:SolrCloudSnapshots_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40149/solr/SolrCloudSnapshots_shard2_replica_n6/ shard2
[junit4] 2> 2434944 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr c:SolrCloudSnapshots s:shard1 r:core_node3 x:SolrCloudSnapshots_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2434944 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr c:SolrCloudSnapshots s:shard1 r:core_node3 x:SolrCloudSnapshots_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2434945 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr c:SolrCloudSnapshots s:shard1 r:core_node3 x:SolrCloudSnapshots_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:46589/solr/SolrCloudSnapshots_shard1_replica_n1/
[junit4] 2> 2434945 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr c:SolrCloudSnapshots s:shard1 r:core_node3 x:SolrCloudSnapshots_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=SolrCloudSnapshots_shard1_replica_n1 url=http://127.0.0.1:46589/solr START replicas=[http://127.0.0.1:42827/solr/SolrCloudSnapshots_shard1_replica_n2/] nUpdates=100
[junit4] 2> 2434945 INFO (qtp28825050-26272) [n:127.0.0.1:42827_solr c:SolrCloudSnapshots s:shard1 r:core_node5 x:SolrCloudSnapshots_shard1_replica_n2] o.a.s.c.S.Request [SolrCloudSnapshots_shard1_replica_n2] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2435245 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr c:SolrCloudSnapshots s:shard1 r:core_node3 x:SolrCloudSnapshots_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=SolrCloudSnapshots_shard1_replica_n1 url=http://127.0.0.1:46589/solr DONE. We have no versions. sync failed.
[junit4] 2> 2435245 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr c:SolrCloudSnapshots s:shard1 r:core_node3 x:SolrCloudSnapshots_shard1_replica_n1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 2435245 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr c:SolrCloudSnapshots s:shard1 r:core_node3 x:SolrCloudSnapshots_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 2435245 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr c:SolrCloudSnapshots s:shard1 r:core_node3 x:SolrCloudSnapshots_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2435249 INFO (qtp22149981-26288) [n:127.0.0.1:46589_solr c:SolrCloudSnapshots s:shard1 r:core_node3 x:SolrCloudSnapshots_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:46589/solr/SolrCloudSnapshots_shard1_replica_n1/ shard1
[junit4] 2> 2612873 ERROR (qtp28825050-26287) [n:127.0.0.1:42827_solr ] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: create the collection time out:180s
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:314)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:246)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:224)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:735)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:716)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:497)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 2612876 INFO (qtp28825050-26287) [n:127.0.0.1:42827_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=conf1&name=SolrCloudSnapshots&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2} status=500 QTime=180006
[junit4] 2> 2612877 ERROR (qtp28825050-26287) [n:127.0.0.1:42827_solr ] o.a.s.s.HttpSolrCall null:org.apache.solr.common.SolrException: create the collection time out:180s
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:314)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:246)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:224)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:735)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:716)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:497)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 2612878 INFO (TEST-TestSolrCloudSnapshots.testSnapshots-seed#[B4276B5D926F10BF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testSnapshots
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSolrCloudSnapshots -Dtests.method=testSnapshots -Dtests.seed=B4276B5D926F10BF -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=fi-FI -Dtests.timezone=Asia/Choibalsan -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 180s J1 | TestSolrCloudSnapshots.testSnapshots <<<
[junit4] > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:42827/solr: create the collection time out:180s
[junit4] > at __randomizedtesting.SeedInfo.seed([B4276B5D926F10BF:826BEFC2E73612F0]:0)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1103)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:883)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
[junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
[junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
[junit4] > at org.apache.solr.core.snapshots.TestSolrCloudSnapshots.testSnapshots(TestSolrCloudSnapshots.java:86)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2612883 INFO (jetty-closer-4688-thread-2) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@127e5e0{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2612883 INFO (jetty-closer-4688-thread-3) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@984204{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2612885 ERROR (OverseerThreadFactory-7546-thread-1-processing-n:127.0.0.1:40435_solr) [n:127.0.0.1:40435_solr ] o.a.s.c.OverseerCollectionMessageHandler Error from shard: http://127.0.0.1:46589/solr
[junit4] 2> org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:46589/solr
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:657)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
[junit4] 2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
[junit4] 2> at org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:172)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: org.apache.http.NoHttpResponseException: 127.0.0.1:46589 failed to respond
[junit4] 2> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
[junit4] 2> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
[junit4] 2> at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
[junit4] 2> at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
[junit4] 2> at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
[junit4] 2> at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
[junit4] 2> at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
[junit4] 2> at org.apache.solr.util.stats.InstrumentedHttpRequestExecutor.execute(InstrumentedHttpRequestExecutor.java:118)
[junit4] 2> at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
[junit4] 2> at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
[junit4] 2> at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
[junit4] 2> at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
[junit4] 2> at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
[junit4] 2> ... 12 more
[junit4] 2> 2612885 INFO (jetty-closer-4688-thread-4) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@179532c{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2612888 INFO (jetty-closer-4688-thread-4) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=27679355
[junit4] 2> 2612884 INFO (jetty-closer-4688-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@67eb3f{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2612888 INFO (jetty-closer-4688-thread-4) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 2612889 INFO (jetty-closer-4688-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=24671129
[junit4] 2> 2612889 INFO (jetty-closer-4688-thread-4) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1e42821: rootName = solr_40149, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@f1b4be
[junit4] 2> 2612889 INFO (jetty-closer-4688-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 2612884 INFO (jetty-closer-4688-thread-2) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=28526663
[junit4] 2> 2612889 INFO (jetty-closer-4688-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@b60a4f: rootName = solr_46589, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@c1371d
[junit4] 2> 2612889 INFO (jetty-closer-4688-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 2612889 INFO (jetty-closer-4688-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@b1ef64: rootName = solr_42827, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@945b5e
[junit4] 2> 2612888 ERROR (OverseerThreadFactory-7546-thread-1-processing-n:127.0.0.1:40435_solr) [n:127.0.0.1:40435_solr ] o.a.s.c.OverseerCollectionMessageHandler Error from shard: http://127.0.0.1:40435/solr
[junit4] 2> org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:40435/solr
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:657)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
[junit4] 2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
[junit4] 2> at org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:172)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: org.apache.http.NoHttpResponseException: 127.0.0.1:40435 failed to respond
[junit4] 2> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
[junit4] 2> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
[junit4] 2> at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
[junit4] 2> at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
[junit4] 2> at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
[junit4] 2> at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
[junit4] 2> at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
[junit4] 2> at org.apache.solr.util.stats.InstrumentedHttpRequestExecutor.execute(InstrumentedHttpRequestExecutor.java:118)
[junit4] 2> at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
[junit4] 2> at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
[junit4] 2> at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
[junit4] 2> at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
[junit4] 2> at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
[junit4] 2> ... 12 more
[junit4] 2> 2612892 ERROR (OverseerThreadFactory-7546-thread-1-processing-n:127.0.0.1:40435_solr) [n:127.0.0.1:40435_solr ] o.a.s.c.OverseerCollectionMessageHandler Error from shard: http://127.0.0.1:42827/solr
[junit4] 2> org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:42827/solr
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:657)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
[junit4] 2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
[junit4] 2> at org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:172)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: org.apache.http.NoHttpResponseException: 127.0.0.1:42827 failed to respond
[junit4] 2> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
[junit4] 2> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
[junit4] 2> at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
[junit4] 2> at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
[junit4] 2> at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
[junit4] 2> at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
[junit4] 2> at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
[junit4] 2> at org.apache.solr.util.stats.InstrumentedHttpRequestExecutor.execute(InstrumentedHttpRequestExecutor.java:118)
[junit4] 2> at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
[junit4] 2> at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
[junit4] 2> at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
[junit4] 2> at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
[junit4] 2> at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
[junit4] 2> ... 12 more
[junit4] 2> 2612886 INFO (jetty-closer-4688-thread-3) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=15175699
[junit4] 2> 2612894 ERROR (OverseerThreadFactory-7546-thread-1-processing-n:127.0.0.1:40435_solr) [n:127.0.0.1:40435_solr ] o.a.s.c.OverseerCollectionMessageHandler Error from shard: http://127.0.0.1:40149/solr
[junit4] 2> org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:40149/solr
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:657)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
[junit4] 2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
[junit4] 2> at org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:172)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: org.apache.http.NoHttpResponseException: 127.0.0.1:40149 failed to respond
[junit4] 2> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
[junit4] 2> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
[junit4] 2> at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
[junit4] 2> at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
[junit4] 2> at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
[junit4] 2> at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
[junit4] 2> at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
[junit4] 2> at org.apache.solr.util.stats.InstrumentedHttpRequestExecutor.execute(InstrumentedHttpRequestExecutor.java:118)
[junit4] 2> at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
[junit4] 2> at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
[junit4] 2> at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
[junit4] 2> at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
[junit4] 2> at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
[junit4] 2> ... 12 more
[junit4] 2> 2612895 INFO (jetty-closer-4688-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 2612896 ERROR (OverseerThreadFactory-7546-thread-1-processing-n:127.0.0.1:40435_solr) [n:127.0.0.1:40435_solr ] o.a.s.c.OverseerCollectionMessageHandler Cleaning up collection [SolrCloudSnapshots].
[junit4] 2> 2612897 INFO (jetty-closer-4688-thread-3) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1c66d4b: rootName = solr_40435, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@10524d8
[junit4] 2> 2612897 INFO (OverseerThreadFactory-7546-thread-1-processing-n:127.0.0.1:40435_solr) [n:127.0.0.1:40435_solr ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
[junit4] 2> 2612899 ERROR (OverseerThreadFactory-7546-thread-1-processing-n:127.0.0.1:40435_solr) [n:127.0.0.1:40435_solr ] o.a.s.c.OverseerCollectionMessageHandler Error from shard: http://127.0.0.1:46589/solr
[junit4] 2> org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:46589/solr
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:650)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
[junit4] 2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
[junit4] 2> at org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:172)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: org.apache.http.conn.HttpHostConnectException: Connect to 127.0.0.1:46589 [/127.0.0.1] failed: Connection refused (Connection refused)
[junit4] 2> at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159)
[junit4] 2> at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359)
[junit4] 2> at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381)
[junit4] 2> at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
[junit4] 2> at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
[junit4] 2> at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
[junit4] 2> at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
[junit4] 2> at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
[junit4] 2> ... 12 more
[junit4] 2> Caused by: java.net.ConnectException: Connection refused (Connection refused)
[junit4] 2> at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4] 2> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
[junit4] 2> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
[junit4] 2> at java.net.AbstractPlainSocketImpl.connect(Abstrac
[...truncated too long message...]
g.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: org.apache.solr.common.SolrException: No registered leader was found after waiting for 1220000ms , collection: SolrCloudSnapshots slice: shard2 saw state=null with live_nodes=[]
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:772)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:737)
[junit4] 2> at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1178)
[junit4] 2> ... 41 more
[junit4] 2>
[junit4] 2> 2642987 INFO (qtp21827506-26271) [n:127.0.0.1:40435_solr c:SolrCloudSnapshots s:shard2 r:core_node7 x:SolrCloudSnapshots_shard2_replica_n4] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=SolrCloudSnapshots_shard2_replica_n4&action=CREATE&numShards=2&collection=SolrCloudSnapshots&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=209807
[junit4] 2> 2642987 INFO (jetty-closer-4688-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5b93e{/solr,null,UNAVAILABLE}
[junit4] 2> 2642988 ERROR (SUITE-TestSolrCloudSnapshots-seed#[B4276B5D926F10BF]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2642988 INFO (SUITE-TestSolrCloudSnapshots-seed#[B4276B5D926F10BF]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46189 46189
[junit4] 2> 2643108 INFO (Thread-10083) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46189 46189
[junit4] 2> 2643109 WARN (Thread-10083) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/aliases.json
[junit4] 2> 5 /solr/clusterprops.json
[junit4] 2> 4 /solr/security.json
[junit4] 2> 4 /solr/configs/conf1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 12 /solr/collections/SolrCloudSnapshots/state.json
[junit4] 2> 5 /solr/clusterstate.json
[junit4] 2> 2 /solr/overseer_elect/election/99138541887684620-127.0.0.1:40435_solr-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/live_nodes
[junit4] 2> 5 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.snapshots.TestSolrCloudSnapshots_B4276B5D926F10BF-001
[junit4] 2> Dec 08, 2017 11:58:29 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=428, maxMBSortInHeap=5.18225074266071, sim=RandomSimilarity(queryNorm=true): {}, locale=fi-FI, timezone=Asia/Choibalsan
[junit4] 2> NOTE: Linux 4.10.0-40-generic i386/Oracle Corporation 1.8.0_144 (32-bit)/cpus=8,threads=1,free=186267872,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [TestSolrCloudWithKerberosAlt, ChaosMonkeySafeLeaderTest, DistribDocExpirationUpdateProcessorTest, TestSolrCoreParser, TestExclusionRuleCollectionAccess, CoreAdminCreateDiscoverTest, RegexBoostProcessorTest, AddReplicaTest, BitVectorTest, DistributedTermsComponentTest, TestStressUserVersions, HttpSolrCallGetCoreTest, CdcrUpdateLogTest, SolrCLIZkUtilsTest, PeerSyncReplicationTest, TestSchemaManager, TestSchemaSimilarityResource, LeaderFailureAfterFreshStartTest, TestManagedSchemaAPI, TestCursorMarkWithoutUniqueKey, TestRandomCollapseQParserPlugin, FacetPivotSmallTest, RankQueryTest, ZkSolrClientTest, TestLegacyFieldCache, ConcurrentDeleteAndCreateCollectionTest, TestSlowCompositeReaderWrapper, DocumentAnalysisRequestHandlerTest, TestRuleBasedAuthorizationPlugin, DistributedFacetExistsSmallTest, HdfsUnloadDistributedZkTest, TestStressRecovery, TestUseDocValuesAsStored2, JSONWriterTest, TestXmlQParserPlugin, DeleteLastCustomShardedReplicaTest, TestTolerantUpdateProcessorCloud, ImplicitSnitchTest, TestStreamBody, HLLSerializationTest, CollectionTooManyReplicasTest, TestCopyFieldCollectionResource, CloudExitableDirectoryReaderTest, TestSolrCLIRunExample, ShufflingReplicaListTransformerTest, ClassificationUpdateProcessorIntegrationTest, ShardRoutingCustomTest, BinaryUpdateRequestHandlerTest, TestManagedSynonymGraphFilterFactory, SubstringBytesRefFilterTest, TestLegacyNumericUtils, PolyFieldTest, NotRequiredUniqueKeyTest, CdcrReplicationHandlerTest, BadIndexSchemaTest, CdcrBidirectionalTest, TestReplicaProperties, BlockJoinFacetSimpleTest, SolrJmxReporterCloudTest, TestStressReorder, TestFaceting, CheckHdfsIndexTest, JavabinLoaderTest, TestCustomStream, PreAnalyzedFieldManagedSchemaCloudTest, DocValuesMultiTest, TestAnalyzedSuggestions, ReplaceNodeTest, TestMinMaxOnMultiValuedField, TestBulkSchemaConcurrent, TestFieldCollectionResource, TestBackupRepositoryFactory, MetricUtilsTest, TestCollationField, ShowFileRequestHandlerTest, SolrMetricsIntegrationTest, PreAnalyzedUpdateProcessorTest, TestCollectionAPI, TestHdfsUpdateLog, PreAnalyzedFieldTest, V2ApiIntegrationTest, SOLR749Test, TestRTGBase, SuggesterFSTTest, TestNamedUpdateProcessors, TestRecoveryHdfs, DocExpirationUpdateProcessorFactoryTest, TestSurroundQueryParser, TestSolr4Spatial, SuggesterTSTTest, TestDocSet, TestSimpleQParserPlugin, UUIDUpdateProcessorFallbackTest, TestSolrQueryResponse, TestDistribIDF, TestConfigSetsAPIExclusivity, TestStressInPlaceUpdates, TestLRUStatsCache, TestImpersonationWithHadoopAuth, CircularListTest, TestCustomDocTransformer, DocValuesMissingTest, TestPolicyCloud, TestRandomFlRTGCloud, HdfsSyncSliceTest, TestRemoteStreaming, TestAtomicUpdateErrorCases, SolrIndexSplitterTest, OpenCloseCoreStressTest, TestStressLucene, TestGraphTermsQParserPlugin, TestSubQueryTransformerCrossCore, TestMultiWordSynonyms, TestManagedResourceStorage, TestUniqueKeyFieldResource, TestLeaderElectionZkExpiry, ConfigureRecoveryStrategyTest, ChangedSchemaMergeTest, PingRequestHandlerTest, TestGraphMLResponseWriter, TestFastLRUCache, MBeansHandlerTest, CollectionsAPIDistributedZkTest, OverseerTest, LeaderElectionIntegrationTest, ShardRoutingTest, BasicZkTest, RecoveryZkTest, TestRandomFaceting, ZkCLITest, TestStressVersions, TestCoreContainer, SimpleFacetsTest, StatsComponentTest, ConvertedLegacyTest, BasicFunctionalityTest, TestLazyCores, SoftAutoCommitTest, AssignTest, CleanupOldIndexTest, CollectionsAPIAsyncDistributedZkTest, DeleteInactiveReplicaTest, DeleteReplicaTest, DistribJoinFromCollectionTest, DistributedVersionInfoTest, LeaderElectionContextKeyTest, LeaderFailoverAfterPartitionTest, LeaderInitiatedRecoveryOnShardRestartTest, LegacyCloudClusterPropTest, MigrateRouteKeyTest, NodeMutatorTest, OutOfBoxZkACLAndCredentialsProvidersTest, OverriddenZkACLAndCredentialsProvidersTest, OverseerRolesTest, OverseerTaskQueueTest, RollingRestartTest, TestAuthenticationFramework, TestCloudRecovery, TestConfigSetsAPI, TestPrepRecovery, TestSSLRandomization, TestShortCircuitedRequests, TestSizeLimitedDistributedMap, TestSolrCloudWithDelegationTokens, TestStressCloudBlindAtomicUpdates, TlogReplayBufferedWhileIndexingTest, TriLevelCompositeIdRoutingTest, AutoAddReplicasIntegrationTest, HdfsAutoAddReplicasIntegrationTest, CdcrReplicationDistributedZkTest, HdfsBasicDistributedZk2Test, HdfsChaosMonkeySafeLeaderTest, HdfsCollectionsAPIDistributedZkTest, HdfsNNFailoverTest, HdfsRecoverLeaseTest, HdfsThreadLeakTest, HdfsTlogReplayBufferedWhileIndexingTest, TestClusterStateMutator, ZkStateWriterTest, CachingDirectoryFactoryTest, ExitableDirectoryReaderTest, HdfsDirectoryFactoryTest, TestImplicitCoreProperties, TestInfoStreamLogging, TestInitParams, TestReloadAndDeleteDocs, TestShardHandlerFactory, TestSolrConfigHandler, TestSolrCloudSnapshots]
[junit4] Completed [685/751 (1!)] on J1 in 211.49s, 1 test, 1 error <<< FAILURES!
[...truncated 194 lines...]
[junit4] JVM J0: stdout was not empty, see: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/temp/junit4-J0-20171208_111424_6371519290188601365710.sysout
[junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: Java heap space
[junit4] Dumping heap to /home/jenkins/workspace/Lucene-Solr-7.x-Linux/heapdumps/java_pid4748.hprof ...
[junit4] Heap dump file created [300906469 bytes in 0.461 secs]
[junit4] <<< JVM J0: EOF ----
[...truncated 8490 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/build.xml:835: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/build.xml:787: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid4748.hprof
Total time: 73 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-7.x-Linux (64bit/jdk-9.0.1) - Build # 957 -
Still unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/957/
Java: 64bit/jdk-9.0.1 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
4 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.ltr.TestLTROnSolrCloud
Error Message:
51 threads leaked from SUITE scope at org.apache.solr.ltr.TestLTROnSolrCloud: 1) Thread[id=194, name=Scheduler-103009475, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method) at java.base@9.0.1/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2104) at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1131) at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:848) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 2) Thread[id=164, name=jetty-launcher-13-thread-2-EventThread, state=WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method) at java.base@9.0.1/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062) at java.base@9.0.1/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435) at app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501) 3) Thread[id=170, name=OverseerCollectionConfigSetProcessor-99137650166464518-127.0.0.1:44011_solr-n_0000000000, state=TIMED_WAITING, group=Overseer collection creation process.] at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method) at java.base@9.0.1/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2104) at app//org.apache.solr.cloud.ZkDistributedQueue.peekElements(ZkDistributedQueue.java:435) at app//org.apache.solr.cloud.OverseerTaskQueue.peekTopN(OverseerTaskQueue.java:235) at app//org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:228) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 4) Thread[id=153, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/java.lang.Thread.sleep(Native Method) at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 5) Thread[id=198, name=ScheduledTrigger-88-thread-4, state=WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method) at java.base@9.0.1/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062) at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1126) at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:848) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 6) Thread[id=132, name=qtp1669448773-132, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:265) at java.base@9.0.1/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92) at java.base@9.0.1/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at app//org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 7) Thread[id=197, name=ScheduledTrigger-88-thread-3, state=WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method) at java.base@9.0.1/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062) at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1126) at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:848) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 8) Thread[id=117, name=Thread-33, state=WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/java.lang.Object.wait(Native Method) at java.base@9.0.1/java.lang.Thread.join(Thread.java:1353) at java.base@9.0.1/java.lang.Thread.join(Thread.java:1427) at app//org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:297) at app//org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:309) at app//org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:490) 9) Thread[id=127, name=qtp1238970803-127, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/java.lang.Thread.sleep(Native Method) at app//org.apache.solr.cloud.ZkController.waitForShardId(ZkController.java:1562) at app//org.apache.solr.cloud.ZkController.doGetShardIdAndNodeNameProcess(ZkController.java:1509) at app//org.apache.solr.cloud.ZkController.preRegister(ZkController.java:1622) at app//org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1030) at app//org.apache.solr.core.CoreContainer.create(CoreContainer.java:948) at app//org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91) at app//org.apache.solr.handler.admin.CoreAdminOperation$$Lambda$198/1660819750.execute(Unknown Source) at app//org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384) at app//org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389) at app//org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174) at app//org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177) at app//org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:735) at app//org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:716) at app//org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:497) at app//org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382) at app//org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326) at app//org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) at app//org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at app//org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) at app//org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at app//org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at app//org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at app//org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at app//org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at app//org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at app//org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at app//org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426) at app//org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at app//org.eclipse.jetty.server.Server.handle(Server.java:534) at app//org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at app//org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at app//org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) at app//org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) at app//org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251) at app//org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) at app//org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) at app//org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 10) Thread[id=136, name=qtp1669448773-136, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/java.lang.Thread.sleep(Native Method) at app//org.apache.solr.cloud.ZkController.waitForShardId(ZkController.java:1562) at app//org.apache.solr.cloud.ZkController.doGetShardIdAndNodeNameProcess(ZkController.java:1509) at app//org.apache.solr.cloud.ZkController.preRegister(ZkController.java:1622) at app//org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1030) at app//org.apache.solr.core.CoreContainer.create(CoreContainer.java:948) at app//org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91) at app//org.apache.solr.handler.admin.CoreAdminOperation$$Lambda$198/1660819750.execute(Unknown Source) at app//org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384) at app//org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389) at app//org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174) at app//org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177) at app//org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:735) at app//org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:716) at app//org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:497) at app//org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382) at app//org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326) at app//org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) at app//org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at app//org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) at app//org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at app//org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at app//org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at app//org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at app//org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at app//org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at app//org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at app//org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426) at app//org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at app//org.eclipse.jetty.server.Server.handle(Server.java:534) at app//org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at app//org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at app//org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) at app//org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) at app//org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251) at app//org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) at app//org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) at app//org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 11) Thread[id=151, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/java.lang.Thread.sleep(Native Method) at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 12) Thread[id=137, name=org.eclipse.jetty.server.session.HashSessionManager@7f92c044Timer, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method) at java.base@9.0.1/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2104) at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1131) at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:848) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 13) Thread[id=200, name=zkCallback-31-thread-3, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method) at java.base@9.0.1/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@9.0.1/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462) at java.base@9.0.1/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361) at java.base@9.0.1/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 14) Thread[id=165, name=zkConnectionManagerCallback-30-thread-1, state=WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method) at java.base@9.0.1/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062) at java.base@9.0.1/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 15) Thread[id=172, name=OverseerAutoScalingTriggerThread-99137650166464518-127.0.0.1:44011_solr-n_0000000000, state=WAITING, group=Overseer autoscaling triggers] at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method) at java.base@9.0.1/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062) at app//org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:166) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 16) Thread[id=130, name=qtp1669448773-130, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method) at java.base@9.0.1/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2104) at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 17) Thread[id=163, name=jetty-launcher-13-thread-2-SendThread(127.0.0.1:35417), state=RUNNABLE, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:265) at java.base@9.0.1/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92) at java.base@9.0.1/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349) at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141) 18) Thread[id=176, name=Thread-50, state=WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/java.lang.Object.wait(Native Method) at java.base@9.0.1/java.lang.Object.wait(Object.java:516) at app//org.apache.solr.core.CloserThread.run(CoreContainer.java:1717) 19) Thread[id=129, name=qtp1669448773-129, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:265) at java.base@9.0.1/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92) at java.base@9.0.1/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at app//org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 20) Thread[id=187, name=TEST-TestLTROnSolrCloud.testSimpleQuery-seed#[339FE89529BF798F]-EventThread, state=WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method) at java.base@9.0.1/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062) at java.base@9.0.1/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435) at app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501) 21) Thread[id=161, name=jetty-launcher-13-thread-1-SendThread(127.0.0.1:35417), state=RUNNABLE, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:265) at java.base@9.0.1/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92) at java.base@9.0.1/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349) at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141) 22) Thread[id=144, name=org.eclipse.jetty.server.session.HashSessionManager@4d83c543Timer, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method) at java.base@9.0.1/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2104) at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1131) at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:848) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 23) Thread[id=186, name=TEST-TestLTROnSolrCloud.testSimpleQuery-seed#[339FE89529BF798F]-SendThread(127.0.0.1:35417), state=RUNNABLE, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:265) at java.base@9.0.1/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92) at java.base@9.0.1/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349) at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141) 24) Thread[id=182, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/java.lang.Thread.sleep(Native Method) at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 25) Thread[id=131, name=qtp1669448773-131, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:265) at java.base@9.0.1/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92) at java.base@9.0.1/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 26) Thread[id=188, name=zkConnectionManagerCallback-43-thread-1, state=WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method) at java.base@9.0.1/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062) at java.base@9.0.1/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 27) Thread[id=189, name=Scheduler-1467281307, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method) at java.base@9.0.1/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2104) at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1131) at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:848) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 28) Thread[id=138, name=qtp1238970803-138, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:265) at java.base@9.0.1/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92) at java.base@9.0.1/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at app//org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 29) Thread[id=143, name=qtp1238970803-143, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method) at java.base@9.0.1/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2104) at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 30) Thread[id=152, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/java.lang.Thread.sleep(Native Method) at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 31) Thread[id=141, name=qtp1238970803-141, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:265) at java.base@9.0.1/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92) at java.base@9.0.1/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 32) Thread[id=139, name=qtp1238970803-139, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:265) at java.base@9.0.1/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92) at java.base@9.0.1/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at app//org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 33) Thread[id=154, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/java.lang.Thread.sleep(Native Method) at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 34) Thread[id=173, name=ScheduledTrigger-88-thread-1, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method) at java.base@9.0.1/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2104) at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1131) at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:848) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 35) Thread[id=190, name=OverseerThreadFactory-90-thread-1-processing-n:127.0.0.1:44011_solr, state=WAITING, group=Overseer collection creation process.] at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method) at java.base@9.0.1/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062) at java.base@9.0.1/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435) at java.base@9.0.1/java.util.concurrent.ExecutorCompletionService.take(ExecutorCompletionService.java:192) at app//org.apache.solr.handler.component.HttpShardHandler.take(HttpShardHandler.java:248) at app//org.apache.solr.handler.component.HttpShardHandler.takeCompletedOrError(HttpShardHandler.java:241) at app//org.apache.solr.cloud.OverseerCollectionMessageHandler.processResponses(OverseerCollectionMessageHandler.java:728) at app//org.apache.solr.cloud.CreateCollectionCmd.call(CreateCollectionCmd.java:297) at app//org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:233) at app//org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:464) at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188) at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$232/1278920200.run(Unknown Source) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 36) Thread[id=134, name=qtp1669448773-134-acceptor-0@38a43d08-ServerConnector@569f2276{SSL,[ssl, http/1.1]}{127.0.0.1:44011}, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at java.base@9.0.1/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:424) at java.base@9.0.1/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:252) at app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371) at app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 37) Thread[id=133, name=qtp1669448773-133, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method) at java.base@9.0.1/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2104) at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 38) Thread[id=140, name=qtp1238970803-140-acceptor-0@438b327e-ServerConnector@6b19d87b{SSL,[ssl, http/1.1]}{127.0.0.1:35683}, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at java.base@9.0.1/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:424) at java.base@9.0.1/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:252) at app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371) at app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 39) Thread[id=128, name=qtp1238970803-128, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:265) at java.base@9.0.1/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92) at java.base@9.0.1/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at app//org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 40) Thread[id=166, name=zkConnectionManagerCallback-32-thread-1, state=WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method) at java.base@9.0.1/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062) at java.base@9.0.1/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 41) Thread[id=118, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:265) at java.base@9.0.1/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92) at java.base@9.0.1/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at app//org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:173) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 42) Thread[id=171, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/java.lang.Thread.sleep(Native Method) at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 43) Thread[id=135, name=qtp1669448773-135, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:265) at java.base@9.0.1/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92) at java.base@9.0.1/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 44) Thread[id=199, name=zkCallback-31-thread-2, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method) at java.base@9.0.1/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@9.0.1/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462) at java.base@9.0.1/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361) at java.base@9.0.1/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 45) Thread[id=121, name=ProcessThread(sid:0 cport:35417):, state=WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method) at java.base@9.0.1/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062) at java.base@9.0.1/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435) at app//org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:122) 46) Thread[id=162, name=jetty-launcher-13-thread-1-EventThread, state=WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method) at java.base@9.0.1/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062) at java.base@9.0.1/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435) at app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501) 47) Thread[id=169, name=OverseerStateUpdate-99137650166464518-127.0.0.1:44011_solr-n_0000000000, state=TIMED_WAITING, group=Overseer state updater.] at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method) at java.base@9.0.1/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2104) at app//org.apache.solr.cloud.ZkDistributedQueue.peekElements(ZkDistributedQueue.java:435) at app//org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:203) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 48) Thread[id=119, name=SessionTracker, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/java.lang.Object.wait(Native Method) at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146) 49) Thread[id=142, name=qtp1238970803-142, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method) at java.base@9.0.1/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2104) at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844) 50) Thread[id=175, name=Thread-51, state=WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/java.lang.Object.wait(Native Method) at java.base@9.0.1/java.lang.Object.wait(Object.java:516) at app//org.apache.solr.core.CloserThread.run(CoreContainer.java:1717) 51) Thread[id=174, name=ScheduledTrigger-88-thread-2, state=WAITING, group=TGRP-TestLTROnSolrCloud] at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method) at java.base@9.0.1/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062) at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1126) at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:848) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152) at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 51 threads leaked from SUITE scope at org.apache.solr.ltr.TestLTROnSolrCloud:
1) Thread[id=194, name=Scheduler-103009475, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method)
at java.base@9.0.1/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2104)
at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1131)
at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:848)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
2) Thread[id=164, name=jetty-launcher-13-thread-2-EventThread, state=WAITING, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method)
at java.base@9.0.1/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062)
at java.base@9.0.1/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
at app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
3) Thread[id=170, name=OverseerCollectionConfigSetProcessor-99137650166464518-127.0.0.1:44011_solr-n_0000000000, state=TIMED_WAITING, group=Overseer collection creation process.]
at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method)
at java.base@9.0.1/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2104)
at app//org.apache.solr.cloud.ZkDistributedQueue.peekElements(ZkDistributedQueue.java:435)
at app//org.apache.solr.cloud.OverseerTaskQueue.peekTopN(OverseerTaskQueue.java:235)
at app//org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:228)
at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
4) Thread[id=153, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/java.lang.Thread.sleep(Native Method)
at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
5) Thread[id=198, name=ScheduledTrigger-88-thread-4, state=WAITING, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method)
at java.base@9.0.1/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062)
at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1126)
at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:848)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
6) Thread[id=132, name=qtp1669448773-132, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:265)
at java.base@9.0.1/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92)
at java.base@9.0.1/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
at app//org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
7) Thread[id=197, name=ScheduledTrigger-88-thread-3, state=WAITING, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method)
at java.base@9.0.1/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062)
at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1126)
at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:848)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
8) Thread[id=117, name=Thread-33, state=WAITING, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/java.lang.Object.wait(Native Method)
at java.base@9.0.1/java.lang.Thread.join(Thread.java:1353)
at java.base@9.0.1/java.lang.Thread.join(Thread.java:1427)
at app//org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:297)
at app//org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:309)
at app//org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:490)
9) Thread[id=127, name=qtp1238970803-127, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/java.lang.Thread.sleep(Native Method)
at app//org.apache.solr.cloud.ZkController.waitForShardId(ZkController.java:1562)
at app//org.apache.solr.cloud.ZkController.doGetShardIdAndNodeNameProcess(ZkController.java:1509)
at app//org.apache.solr.cloud.ZkController.preRegister(ZkController.java:1622)
at app//org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1030)
at app//org.apache.solr.core.CoreContainer.create(CoreContainer.java:948)
at app//org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
at app//org.apache.solr.handler.admin.CoreAdminOperation$$Lambda$198/1660819750.execute(Unknown Source)
at app//org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
at app//org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
at app//org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
at app//org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
at app//org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:735)
at app//org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:716)
at app//org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:497)
at app//org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
at app//org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
at app//org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at app//org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
at app//org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at app//org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at app//org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at app//org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at app//org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at app//org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at app//org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at app//org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at app//org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426)
at app//org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at app//org.eclipse.jetty.server.Server.handle(Server.java:534)
at app//org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at app//org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at app//org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at app//org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at app//org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251)
at app//org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at app//org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at app//org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
10) Thread[id=136, name=qtp1669448773-136, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/java.lang.Thread.sleep(Native Method)
at app//org.apache.solr.cloud.ZkController.waitForShardId(ZkController.java:1562)
at app//org.apache.solr.cloud.ZkController.doGetShardIdAndNodeNameProcess(ZkController.java:1509)
at app//org.apache.solr.cloud.ZkController.preRegister(ZkController.java:1622)
at app//org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1030)
at app//org.apache.solr.core.CoreContainer.create(CoreContainer.java:948)
at app//org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
at app//org.apache.solr.handler.admin.CoreAdminOperation$$Lambda$198/1660819750.execute(Unknown Source)
at app//org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
at app//org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
at app//org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
at app//org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
at app//org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:735)
at app//org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:716)
at app//org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:497)
at app//org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
at app//org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
at app//org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at app//org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
at app//org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at app//org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at app//org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at app//org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at app//org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at app//org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at app//org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at app//org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at app//org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426)
at app//org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at app//org.eclipse.jetty.server.Server.handle(Server.java:534)
at app//org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at app//org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at app//org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at app//org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at app//org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251)
at app//org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at app//org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at app//org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
11) Thread[id=151, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/java.lang.Thread.sleep(Native Method)
at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
12) Thread[id=137, name=org.eclipse.jetty.server.session.HashSessionManager@7f92c044Timer, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method)
at java.base@9.0.1/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2104)
at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1131)
at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:848)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
13) Thread[id=200, name=zkCallback-31-thread-3, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method)
at java.base@9.0.1/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
at java.base@9.0.1/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
at java.base@9.0.1/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
at java.base@9.0.1/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
14) Thread[id=165, name=zkConnectionManagerCallback-30-thread-1, state=WAITING, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method)
at java.base@9.0.1/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062)
at java.base@9.0.1/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
15) Thread[id=172, name=OverseerAutoScalingTriggerThread-99137650166464518-127.0.0.1:44011_solr-n_0000000000, state=WAITING, group=Overseer autoscaling triggers]
at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method)
at java.base@9.0.1/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062)
at app//org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:166)
at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
16) Thread[id=130, name=qtp1669448773-130, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method)
at java.base@9.0.1/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2104)
at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
17) Thread[id=163, name=jetty-launcher-13-thread-2-SendThread(127.0.0.1:35417), state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:265)
at java.base@9.0.1/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92)
at java.base@9.0.1/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
18) Thread[id=176, name=Thread-50, state=WAITING, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/java.lang.Object.wait(Native Method)
at java.base@9.0.1/java.lang.Object.wait(Object.java:516)
at app//org.apache.solr.core.CloserThread.run(CoreContainer.java:1717)
19) Thread[id=129, name=qtp1669448773-129, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:265)
at java.base@9.0.1/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92)
at java.base@9.0.1/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
at app//org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
20) Thread[id=187, name=TEST-TestLTROnSolrCloud.testSimpleQuery-seed#[339FE89529BF798F]-EventThread, state=WAITING, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method)
at java.base@9.0.1/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062)
at java.base@9.0.1/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
at app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
21) Thread[id=161, name=jetty-launcher-13-thread-1-SendThread(127.0.0.1:35417), state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:265)
at java.base@9.0.1/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92)
at java.base@9.0.1/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
22) Thread[id=144, name=org.eclipse.jetty.server.session.HashSessionManager@4d83c543Timer, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method)
at java.base@9.0.1/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2104)
at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1131)
at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:848)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
23) Thread[id=186, name=TEST-TestLTROnSolrCloud.testSimpleQuery-seed#[339FE89529BF798F]-SendThread(127.0.0.1:35417), state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:265)
at java.base@9.0.1/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92)
at java.base@9.0.1/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
24) Thread[id=182, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/java.lang.Thread.sleep(Native Method)
at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
25) Thread[id=131, name=qtp1669448773-131, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:265)
at java.base@9.0.1/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92)
at java.base@9.0.1/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
26) Thread[id=188, name=zkConnectionManagerCallback-43-thread-1, state=WAITING, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method)
at java.base@9.0.1/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062)
at java.base@9.0.1/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
27) Thread[id=189, name=Scheduler-1467281307, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method)
at java.base@9.0.1/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2104)
at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1131)
at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:848)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
28) Thread[id=138, name=qtp1238970803-138, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:265)
at java.base@9.0.1/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92)
at java.base@9.0.1/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
at app//org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
29) Thread[id=143, name=qtp1238970803-143, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method)
at java.base@9.0.1/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2104)
at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
30) Thread[id=152, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/java.lang.Thread.sleep(Native Method)
at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
31) Thread[id=141, name=qtp1238970803-141, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:265)
at java.base@9.0.1/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92)
at java.base@9.0.1/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
32) Thread[id=139, name=qtp1238970803-139, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at java.base@9.0.1/sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:265)
at java.base@9.0.1/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92)
at java.base@9.0.1/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at java.base@9.0.1/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at app//org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
at app//org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
33) Thread[id=154, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/java.lang.Thread.sleep(Native Method)
at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
34) Thread[id=173, name=ScheduledTrigger-88-thread-1, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method)
at java.base@9.0.1/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2104)
at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1131)
at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:848)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
35) Thread[id=190, name=OverseerThreadFa
[...truncated too long message...]
ethod)
[junit4] > at app//org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] > at app//org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
[junit4] > at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
[junit4] > 18) Thread[id=175, name=Thread-51, state=WAITING, group=TGRP-TestLTROnSolrCloud]
[junit4] > at java.base@9.0.1/java.lang.Object.wait(Native Method)
[junit4] > at java.base@9.0.1/java.lang.Object.wait(Object.java:516)
[junit4] > at app//org.apache.solr.core.CloserThread.run(CoreContainer.java:1717)
[junit4] > 19) Thread[id=137, name=org.eclipse.jetty.server.session.HashSessionManager@7f92c044Timer, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
[junit4] > at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method)
[junit4] > at java.base@9.0.1/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
[junit4] > at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2104)
[junit4] > at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1131)
[junit4] > at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:848)
[junit4] > at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092)
[junit4] > at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
[junit4] > at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
[junit4] > at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
[junit4] > 20) Thread[id=173, name=ScheduledTrigger-88-thread-1, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
[junit4] > at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method)
[junit4] > at java.base@9.0.1/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
[junit4] > at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2104)
[junit4] > at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1131)
[junit4] > at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:848)
[junit4] > at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092)
[junit4] > at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
[junit4] > at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
[junit4] > at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
[junit4] > 21) Thread[id=200, name=zkCallback-31-thread-3, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
[junit4] > at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method)
[junit4] > at java.base@9.0.1/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
[junit4] > at java.base@9.0.1/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
[junit4] > at java.base@9.0.1/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
[junit4] > at java.base@9.0.1/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
[junit4] > at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)
[junit4] > at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
[junit4] > at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
[junit4] > at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
[junit4] > 22) Thread[id=188, name=zkConnectionManagerCallback-43-thread-1, state=WAITING, group=TGRP-TestLTROnSolrCloud]
[junit4] > at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method)
[junit4] > at java.base@9.0.1/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
[junit4] > at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062)
[junit4] > at java.base@9.0.1/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
[junit4] > at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092)
[junit4] > at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
[junit4] > at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
[junit4] > at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
[junit4] > 23) Thread[id=189, name=Scheduler-1467281307, state=WAITING, group=TGRP-TestLTROnSolrCloud]
[junit4] > at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method)
[junit4] > at java.base@9.0.1/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
[junit4] > at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062)
[junit4] > at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1119)
[junit4] > at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:848)
[junit4] > at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092)
[junit4] > at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
[junit4] > at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
[junit4] > at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
[junit4] > 24) Thread[id=174, name=ScheduledTrigger-88-thread-2, state=WAITING, group=TGRP-TestLTROnSolrCloud]
[junit4] > at java.base@9.0.1/jdk.internal.misc.Unsafe.park(Native Method)
[junit4] > at java.base@9.0.1/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
[junit4] > at java.base@9.0.1/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062)
[junit4] > at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1126)
[junit4] > at java.base@9.0.1/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:848)
[junit4] > at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1092)
[junit4] > at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
[junit4] > at java.base@9.0.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
[junit4] > at java.base@9.0.1/java.lang.Thread.run(Thread.java:844)
[junit4] > at __randomizedtesting.SeedInfo.seed([339FE89529BF798F]:0)
[junit4] Completed [34/35 (1!)] on J0 in 204.26s, 1 test, 3 errors <<< FAILURES!
[...truncated 4 lines...]
[junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/contrib/solr-ltr/test/temp/junit4-J0-20171208_080805_63510615553127394536057.syserr
[junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
[junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[junit4] <<< JVM J0: EOF ----
[...truncated 602 lines...]
[junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/contrib/solr-uima/test/temp/junit4-J1-20171208_081144_60212865858595003971348.syserr
[junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
[junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[junit4] <<< JVM J1: EOF ----
[...truncated 3 lines...]
[junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/contrib/solr-uima/test/temp/junit4-J0-20171208_081144_60215291471586422367073.syserr
[junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
[junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[junit4] <<< JVM J0: EOF ----
[...truncated 548 lines...]
[junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/contrib/solr-velocity/test/temp/junit4-J0-20171208_081155_47016605130102633400664.syserr
[junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
[junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[junit4] <<< JVM J0: EOF ----
[...truncated 31689 lines...]