You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2016/10/07 23:25:33 UTC

[JENKINS] Lucene-Solr-6.x-Solaris (64bit/jdk1.8.0) - Build # 438 - Failure!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Solaris/438/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

All tests passed

Build Log:
[...truncated 53362 lines...]
-ecj-javadoc-lint-src:
    [mkdir] Created dir: /var/tmp/ecj413289492
 [ecj-lint] Compiling 127 source files to /var/tmp/ecj413289492
 [ecj-lint] ----------
 [ecj-lint] 1. ERROR in /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/lucene/benchmark/src/java/org/apache/lucene/benchmark/byTask/tasks/ReadTask.java (at line 21)
 [ecj-lint] 	import java.util.Collection;
 [ecj-lint] 	       ^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import java.util.Collection is never used
 [ecj-lint] ----------
 [ecj-lint] 2. ERROR in /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/lucene/benchmark/src/java/org/apache/lucene/benchmark/byTask/tasks/ReadTask.java (at line 22)
 [ecj-lint] 	import java.util.HashSet;
 [ecj-lint] 	       ^^^^^^^^^^^^^^^^^
 [ecj-lint] The import java.util.HashSet is never used
 [ecj-lint] ----------
 [ecj-lint] 3. ERROR in /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/lucene/benchmark/src/java/org/apache/lucene/benchmark/byTask/tasks/ReadTask.java (at line 24)
 [ecj-lint] 	import java.util.List;
 [ecj-lint] 	       ^^^^^^^^^^^^^^
 [ecj-lint] The import java.util.List is never used
 [ecj-lint] ----------
 [ecj-lint] 4. ERROR in /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/lucene/benchmark/src/java/org/apache/lucene/benchmark/byTask/tasks/ReadTask.java (at line 25)
 [ecj-lint] 	import java.util.Set;
 [ecj-lint] 	       ^^^^^^^^^^^^^
 [ecj-lint] The import java.util.Set is never used
 [ecj-lint] ----------
 [ecj-lint] 5. ERROR in /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/lucene/benchmark/src/java/org/apache/lucene/benchmark/byTask/tasks/ReadTask.java (at line 27)
 [ecj-lint] 	import org.apache.lucene.analysis.Analyzer;
 [ecj-lint] 	       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import org.apache.lucene.analysis.Analyzer is never used
 [ecj-lint] ----------
 [ecj-lint] 6. ERROR in /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/lucene/benchmark/src/java/org/apache/lucene/benchmark/byTask/tasks/ReadTask.java (at line 33)
 [ecj-lint] 	import org.apache.lucene.index.IndexableField;
 [ecj-lint] 	       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import org.apache.lucene.index.IndexableField is never used
 [ecj-lint] ----------
 [ecj-lint] 6 problems (6 errors)

BUILD FAILED
/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/build.xml:765: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/build.xml:101: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/lucene/build.xml:204: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/lucene/common-build.xml:2177: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/lucene/common-build.xml:1992: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/lucene/common-build.xml:2031: Compile failed; see the compiler error output for details.

Total time: 79 minutes 22 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-6.x-Solaris (64bit/jdk1.8.0) - Build # 440 - Still unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Solaris/440/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.cloud.HttpPartitionTest.test

Error Message:
Error from server at http://127.0.0.1:53265/jd/pj/c8n_1x2_shard1_replica1: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 404 </title> </head> <body> <h2>HTTP ERROR: 404</h2> <p>Problem accessing /jd/pj/c8n_1x2_shard1_replica1/update. Reason: <pre>    Not Found</pre></p> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.8.v20160314</a><hr/> </body> </html> 

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at http://127.0.0.1:53265/jd/pj/c8n_1x2_shard1_replica1: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 </title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /jd/pj/c8n_1x2_shard1_replica1/update. Reason:
<pre>    Not Found</pre></p>
<hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.8.v20160314</a><hr/>
</body>
</html>

	at __randomizedtesting.SeedInfo.seed([62308C8551BE9D75:EA64B35FFF42F08D]:0)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:765)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1173)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1062)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1004)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.HttpPartitionTest.sendDoc(HttpPartitionTest.java:609)
	at org.apache.solr.cloud.HttpPartitionTest.sendDoc(HttpPartitionTest.java:595)
	at org.apache.solr.cloud.HttpPartitionTest.testRf2(HttpPartitionTest.java:294)
	at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:125)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:53265/jd/pj/c8n_1x2_shard1_replica1: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 </title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /jd/pj/c8n_1x2_shard1_replica1/update. Reason:
<pre>    Not Found</pre></p>
<hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.8.v20160314</a><hr/>
</body>
</html>

	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:561)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:262)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:435)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:387)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:742)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	... 1 more




Build Log:
[...truncated 12062 lines...]
   [junit4] Suite: org.apache.solr.cloud.HttpPartitionTest
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/init-core-data-001
   [junit4]   2> 2159557 INFO  (SUITE-HttpPartitionTest-seed#[62308C8551BE9D75]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
   [junit4]   2> 2159558 INFO  (SUITE-HttpPartitionTest-seed#[62308C8551BE9D75]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /jd/pj
   [junit4]   2> 2159561 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2159561 INFO  (Thread-5778) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2159561 INFO  (Thread-5778) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2159661 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.ZkTestServer start zk server on port:48344
   [junit4]   2> 2159683 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2159689 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 2159693 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2159697 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2159700 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2159703 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2159706 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2159709 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2159712 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2159717 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2159722 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2159730 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x157a7973afc0001, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2159955 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/control-001/cores/collection1
   [junit4]   2> 2159958 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2159959 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3c4e88f4{/jd/pj,null,AVAILABLE}
   [junit4]   2> 2159961 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6cda89db{HTTP/1.1,[http/1.1]}{127.0.0.1:56111}
   [junit4]   2> 2159962 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.e.j.s.Server Started @2163986ms
   [junit4]   2> 2159962 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/tempDir-001/control/data, hostContext=/jd/pj, hostPort=44510, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/control-001/cores}
   [junit4]   2> 2159963 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 6.3.0
   [junit4]   2> 2159963 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2159963 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2159963 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2016-10-09T03:58:09.036Z
   [junit4]   2> 2159967 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2159967 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/control-001/solr.xml
   [junit4]   2> 2159975 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2159976 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48344/solr
   [junit4]   2> 2160052 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [n:127.0.0.1:44510_jd%2Fpj    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44510_jd%2Fpj
   [junit4]   2> 2160056 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [n:127.0.0.1:44510_jd%2Fpj    ] o.a.s.c.Overseer Overseer (id=96730184983248900-127.0.0.1:44510_jd%2Fpj-n_0000000000) starting
   [junit4]   2> 2160073 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [n:127.0.0.1:44510_jd%2Fpj    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44510_jd%2Fpj
   [junit4]   2> 2160077 INFO  (zkCallback-9631-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj) [n:127.0.0.1:44510_jd%2Fpj    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2160107 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [n:127.0.0.1:44510_jd%2Fpj    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/control-001/cores
   [junit4]   2> 2160107 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [n:127.0.0.1:44510_jd%2Fpj    ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 2160115 INFO  (OverseerStateUpdate-96730184983248900-127.0.0.1:44510_jd%2Fpj-n_0000000000) [n:127.0.0.1:44510_jd%2Fpj    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2161141 WARN  (coreLoadExecutor-10727-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj) [n:127.0.0.1:44510_jd%2Fpj c:control_collection   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2161143 INFO  (coreLoadExecutor-10727-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj) [n:127.0.0.1:44510_jd%2Fpj c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
   [junit4]   2> 2161169 INFO  (coreLoadExecutor-10727-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj) [n:127.0.0.1:44510_jd%2Fpj c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2161337 WARN  (coreLoadExecutor-10727-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj) [n:127.0.0.1:44510_jd%2Fpj c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
   [junit4]   2> 2161340 INFO  (coreLoadExecutor-10727-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj) [n:127.0.0.1:44510_jd%2Fpj c:control_collection   x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2161393 INFO  (coreLoadExecutor-10727-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj) [n:127.0.0.1:44510_jd%2Fpj c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 2161394 INFO  (coreLoadExecutor-10727-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj) [n:127.0.0.1:44510_jd%2Fpj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/control-001/cores/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/control-001/cores/collection1/data/]
   [junit4]   2> 2161394 INFO  (coreLoadExecutor-10727-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj) [n:127.0.0.1:44510_jd%2Fpj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@25685339
   [junit4]   2> 2161396 INFO  (coreLoadExecutor-10727-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj) [n:127.0.0.1:44510_jd%2Fpj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=11, maxMergedSegmentMB=2.611328125, floorSegmentMB=1.349609375, forceMergeDeletesPctAllowed=9.0437645713151, segmentsPerTier=27.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 2161405 WARN  (coreLoadExecutor-10727-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj) [n:127.0.0.1:44510_jd%2Fpj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 2161415 INFO  (coreLoadExecutor-10727-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj) [n:127.0.0.1:44510_jd%2Fpj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2161415 INFO  (coreLoadExecutor-10727-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj) [n:127.0.0.1:44510_jd%2Fpj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2161416 INFO  (coreLoadExecutor-10727-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj) [n:127.0.0.1:44510_jd%2Fpj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2161417 INFO  (coreLoadExecutor-10727-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj) [n:127.0.0.1:44510_jd%2Fpj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2161417 INFO  (coreLoadExecutor-10727-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj) [n:127.0.0.1:44510_jd%2Fpj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=37, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 2161417 INFO  (coreLoadExecutor-10727-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj) [n:127.0.0.1:44510_jd%2Fpj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6713e0b6[collection1] main]
   [junit4]   2> 2161419 INFO  (coreLoadExecutor-10727-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj) [n:127.0.0.1:44510_jd%2Fpj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2161420 INFO  (coreLoadExecutor-10727-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj) [n:127.0.0.1:44510_jd%2Fpj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2161421 INFO  (coreLoadExecutor-10727-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj) [n:127.0.0.1:44510_jd%2Fpj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2161422 INFO  (searcherExecutor-10728-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44510_jd%2Fpj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6713e0b6[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2161425 INFO  (coreLoadExecutor-10727-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj) [n:127.0.0.1:44510_jd%2Fpj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1547682961684430848
   [junit4]   2> 2161454 INFO  (coreZkRegister-10720-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44510_jd%2Fpj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2161455 INFO  (coreZkRegister-10720-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44510_jd%2Fpj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2161455 INFO  (coreZkRegister-10720-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44510_jd%2Fpj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:44510/jd/pj/collection1/
   [junit4]   2> 2161455 INFO  (coreZkRegister-10720-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44510_jd%2Fpj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2161455 INFO  (coreZkRegister-10720-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44510_jd%2Fpj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:44510/jd/pj/collection1/ has no replicas
   [junit4]   2> 2161469 INFO  (coreZkRegister-10720-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44510_jd%2Fpj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:44510/jd/pj/collection1/ shard1
   [junit4]   2> 2161621 INFO  (coreZkRegister-10720-thread-1-processing-n:127.0.0.1:44510_jd%2Fpj x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44510_jd%2Fpj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2161623 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2161625 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2161625 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 2161837 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/shard-1-001/cores/collection1
   [junit4]   2> 2161838 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/shard-1-001
   [junit4]   2> 2161839 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2161841 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@11ed0f34{/jd/pj,null,AVAILABLE}
   [junit4]   2> 2161842 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.e.j.s.ServerConnector Started ServerConnector@5a4610be{HTTP/1.1,[http/1.1]}{127.0.0.1:65023}
   [junit4]   2> 2161843 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.e.j.s.Server Started @2165867ms
   [junit4]   2> 2161843 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/jd/pj, hostPort=38623, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/shard-1-001/cores}
   [junit4]   2> 2161843 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 6.3.0
   [junit4]   2> 2161843 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2161843 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2161844 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2016-10-09T03:58:10.917Z
   [junit4]   2> 2161849 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2161849 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/shard-1-001/solr.xml
   [junit4]   2> 2161857 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2161858 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48344/solr
   [junit4]   2> 2161879 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [n:127.0.0.1:38623_jd%2Fpj    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2161890 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [n:127.0.0.1:38623_jd%2Fpj    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38623_jd%2Fpj
   [junit4]   2> 2161895 INFO  (zkCallback-9631-thread-2-processing-n:127.0.0.1:44510_jd%2Fpj) [n:127.0.0.1:44510_jd%2Fpj    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2161898 INFO  (zkCallback-9635-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2161900 INFO  (zkCallback-9641-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2161923 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [n:127.0.0.1:38623_jd%2Fpj    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/shard-1-001/cores
   [junit4]   2> 2161923 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [n:127.0.0.1:38623_jd%2Fpj    ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 2161930 INFO  (OverseerStateUpdate-96730184983248900-127.0.0.1:44510_jd%2Fpj-n_0000000000) [n:127.0.0.1:44510_jd%2Fpj    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 2162036 INFO  (zkCallback-9641-thread-2-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2162960 WARN  (coreLoadExecutor-10738-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj c:collection1   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2162961 INFO  (coreLoadExecutor-10738-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
   [junit4]   2> 2162974 INFO  (coreLoadExecutor-10738-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2163085 WARN  (coreLoadExecutor-10738-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
   [junit4]   2> 2163087 INFO  (coreLoadExecutor-10738-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj c:collection1   x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2163107 INFO  (coreLoadExecutor-10738-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 2163107 INFO  (coreLoadExecutor-10738-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/shard-1-001/cores/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/shard-1-001/cores/collection1/data/]
   [junit4]   2> 2163108 INFO  (coreLoadExecutor-10738-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@25685339
   [junit4]   2> 2163108 INFO  (coreLoadExecutor-10738-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=11, maxMergedSegmentMB=2.611328125, floorSegmentMB=1.349609375, forceMergeDeletesPctAllowed=9.0437645713151, segmentsPerTier=27.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 2163112 WARN  (coreLoadExecutor-10738-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 2163121 INFO  (coreLoadExecutor-10738-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2163121 INFO  (coreLoadExecutor-10738-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2163122 INFO  (coreLoadExecutor-10738-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2163122 INFO  (coreLoadExecutor-10738-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2163122 INFO  (coreLoadExecutor-10738-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=37, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 2163122 INFO  (coreLoadExecutor-10738-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@37239a48[collection1] main]
   [junit4]   2> 2163124 INFO  (coreLoadExecutor-10738-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2163125 INFO  (coreLoadExecutor-10738-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2163125 INFO  (coreLoadExecutor-10738-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2163126 INFO  (searcherExecutor-10739-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@37239a48[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2163126 INFO  (coreLoadExecutor-10738-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1547682963468058624
   [junit4]   2> 2163138 INFO  (coreZkRegister-10733-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2163138 INFO  (coreZkRegister-10733-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2163138 INFO  (coreZkRegister-10733-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:38623/jd/pj/collection1/
   [junit4]   2> 2163138 INFO  (coreZkRegister-10733-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2163138 INFO  (coreZkRegister-10733-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:38623/jd/pj/collection1/ has no replicas
   [junit4]   2> 2163147 INFO  (coreZkRegister-10733-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:38623/jd/pj/collection1/ shard2
   [junit4]   2> 2163253 INFO  (zkCallback-9641-thread-2-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2163300 INFO  (coreZkRegister-10733-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2163407 INFO  (zkCallback-9641-thread-2-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2163650 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/shard-2-001/cores/collection1
   [junit4]   2> 2163651 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/shard-2-001
   [junit4]   2> 2163652 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2163655 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@32609a05{/jd/pj,null,AVAILABLE}
   [junit4]   2> 2163656 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.e.j.s.ServerConnector Started ServerConnector@13e12c72{HTTP/1.1,[http/1.1]}{127.0.0.1:63879}
   [junit4]   2> 2163656 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.e.j.s.Server Started @2167680ms
   [junit4]   2> 2163656 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/jd/pj, hostPort=43728, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/shard-2-001/cores}
   [junit4]   2> 2163657 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 6.3.0
   [junit4]   2> 2163657 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2163657 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2163657 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2016-10-09T03:58:12.730Z
   [junit4]   2> 2163662 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2163662 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/shard-2-001/solr.xml
   [junit4]   2> 2163668 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2163668 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48344/solr
   [junit4]   2> 2163687 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [n:127.0.0.1:43728_jd%2Fpj    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2163695 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [n:127.0.0.1:43728_jd%2Fpj    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43728_jd%2Fpj
   [junit4]   2> 2163699 INFO  (zkCallback-9631-thread-3-processing-n:127.0.0.1:44510_jd%2Fpj) [n:127.0.0.1:44510_jd%2Fpj    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2163699 INFO  (zkCallback-9635-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2163699 INFO  (zkCallback-9641-thread-2-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2163701 INFO  (zkCallback-9647-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2163721 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [n:127.0.0.1:43728_jd%2Fpj    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/shard-2-001/cores
   [junit4]   2> 2163721 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [n:127.0.0.1:43728_jd%2Fpj    ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 2163726 INFO  (OverseerStateUpdate-96730184983248900-127.0.0.1:44510_jd%2Fpj-n_0000000000) [n:127.0.0.1:44510_jd%2Fpj    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2163831 INFO  (zkCallback-9641-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2163833 INFO  (zkCallback-9647-thread-2-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2164742 WARN  (coreLoadExecutor-10749-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj c:collection1   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2164743 INFO  (coreLoadExecutor-10749-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
   [junit4]   2> 2164755 INFO  (coreLoadExecutor-10749-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2164880 WARN  (coreLoadExecutor-10749-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
   [junit4]   2> 2164883 INFO  (coreLoadExecutor-10749-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj c:collection1   x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2164910 INFO  (coreLoadExecutor-10749-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 2164911 INFO  (coreLoadExecutor-10749-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/shard-2-001/cores/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/shard-2-001/cores/collection1/data/]
   [junit4]   2> 2164911 INFO  (coreLoadExecutor-10749-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@25685339
   [junit4]   2> 2164912 INFO  (coreLoadExecutor-10749-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=11, maxMergedSegmentMB=2.611328125, floorSegmentMB=1.349609375, forceMergeDeletesPctAllowed=9.0437645713151, segmentsPerTier=27.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 2164927 WARN  (coreLoadExecutor-10749-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 2164937 INFO  (coreLoadExecutor-10749-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2164937 INFO  (coreLoadExecutor-10749-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2164937 INFO  (coreLoadExecutor-10749-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2164937 INFO  (coreLoadExecutor-10749-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2164938 INFO  (coreLoadExecutor-10749-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=37, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 2164938 INFO  (coreLoadExecutor-10749-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@389f57f6[collection1] main]
   [junit4]   2> 2164940 INFO  (coreLoadExecutor-10749-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2164940 INFO  (coreLoadExecutor-10749-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2164940 INFO  (coreLoadExecutor-10749-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2164941 INFO  (searcherExecutor-10750-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:43728_jd%2Fpj c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@389f57f6[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2164942 INFO  (coreLoadExecutor-10749-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1547682965371224064
   [junit4]   2> 2164954 INFO  (coreZkRegister-10744-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:43728_jd%2Fpj c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2164955 INFO  (coreZkRegister-10744-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:43728_jd%2Fpj c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2164955 INFO  (coreZkRegister-10744-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:43728_jd%2Fpj c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:43728/jd/pj/collection1/
   [junit4]   2> 2164955 INFO  (coreZkRegister-10744-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:43728_jd%2Fpj c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2164956 INFO  (coreZkRegister-10744-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:43728_jd%2Fpj c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:43728/jd/pj/collection1/ has no replicas
   [junit4]   2> 2164962 INFO  (coreZkRegister-10744-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:43728_jd%2Fpj c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:43728/jd/pj/collection1/ shard1
   [junit4]   2> 2165067 INFO  (zkCallback-9641-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2165067 INFO  (zkCallback-9647-thread-2-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2165115 INFO  (coreZkRegister-10744-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:43728_jd%2Fpj c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2165221 INFO  (zkCallback-9641-thread-1-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2165221 INFO  (zkCallback-9647-thread-2-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2165394 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/shard-3-001/cores/collection1
   [junit4]   2> 2165395 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/shard-3-001
   [junit4]   2> 2165395 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2165397 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2d18a71d{/jd/pj,null,AVAILABLE}
   [junit4]   2> 2165398 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.e.j.s.ServerConnector Started ServerConnector@60bd5c20{HTTP/1.1,[http/1.1]}{127.0.0.1:53634}
   [junit4]   2> 2165398 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.e.j.s.Server Started @2169423ms
   [junit4]   2> 2165398 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/tempDir-001/jetty3, solrconfig=solrconfig.xml, hostContext=/jd/pj, hostPort=53265, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/shard-3-001/cores}
   [junit4]   2> 2165398 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 6.3.0
   [junit4]   2> 2165399 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2165399 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2165399 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2016-10-09T03:58:14.472Z
   [junit4]   2> 2165403 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2165403 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/shard-3-001/solr.xml
   [junit4]   2> 2165408 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2165409 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48344/solr
   [junit4]   2> 2165422 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [n:127.0.0.1:53265_jd%2Fpj    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 2165428 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [n:127.0.0.1:53265_jd%2Fpj    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53265_jd%2Fpj
   [junit4]   2> 2165431 INFO  (zkCallback-9641-thread-2-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2165431 INFO  (zkCallback-9635-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2165431 INFO  (zkCallback-9647-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2165432 INFO  (zkCallback-9631-thread-3-processing-n:127.0.0.1:44510_jd%2Fpj) [n:127.0.0.1:44510_jd%2Fpj    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2165434 INFO  (zkCallback-9653-thread-2-processing-n:127.0.0.1:53265_jd%2Fpj) [n:127.0.0.1:53265_jd%2Fpj    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 2165466 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [n:127.0.0.1:53265_jd%2Fpj    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/shard-3-001/cores
   [junit4]   2> 2165466 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [n:127.0.0.1:53265_jd%2Fpj    ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 2165471 INFO  (OverseerStateUpdate-96730184983248900-127.0.0.1:44510_jd%2Fpj-n_0000000000) [n:127.0.0.1:44510_jd%2Fpj    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 2165581 INFO  (zkCallback-9641-thread-2-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 2165581 INFO  (zkCallback-9647-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 2165581 INFO  (zkCallback-9653-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj) [n:127.0.0.1:53265_jd%2Fpj    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 2166492 WARN  (coreLoadExecutor-10760-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj) [n:127.0.0.1:53265_jd%2Fpj c:collection1   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2166492 INFO  (coreLoadExecutor-10760-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj) [n:127.0.0.1:53265_jd%2Fpj c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
   [junit4]   2> 2166505 INFO  (coreLoadExecutor-10760-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj) [n:127.0.0.1:53265_jd%2Fpj c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2166600 WARN  (coreLoadExecutor-10760-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj) [n:127.0.0.1:53265_jd%2Fpj c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
   [junit4]   2> 2166602 INFO  (coreLoadExecutor-10760-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj) [n:127.0.0.1:53265_jd%2Fpj c:collection1   x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2166620 INFO  (coreLoadExecutor-10760-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj) [n:127.0.0.1:53265_jd%2Fpj c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 2166621 INFO  (coreLoadExecutor-10760-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/shard-3-001/cores/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001/shard-3-001/cores/collection1/data/]
   [junit4]   2> 2166621 INFO  (coreLoadExecutor-10760-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@25685339
   [junit4]   2> 2166622 INFO  (coreLoadExecutor-10760-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=11, maxMergedSegmentMB=2.611328125, floorSegmentMB=1.349609375, forceMergeDeletesPctAllowed=9.0437645713151, segmentsPerTier=27.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 2166627 WARN  (coreLoadExecutor-10760-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 2166637 INFO  (coreLoadExecutor-10760-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2166638 INFO  (coreLoadExecutor-10760-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2166639 INFO  (coreLoadExecutor-10760-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2166639 INFO  (coreLoadExecutor-10760-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2166639 INFO  (coreLoadExecutor-10760-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=37, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 2166639 INFO  (coreLoadExecutor-10760-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@542bbf53[collection1] main]
   [junit4]   2> 2166641 INFO  (coreLoadExecutor-10760-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2166642 INFO  (coreLoadExecutor-10760-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2166642 INFO  (coreLoadExecutor-10760-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2166643 INFO  (searcherExecutor-10761-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@542bbf53[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2166643 INFO  (coreLoadExecutor-10760-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1547682967155900416
   [junit4]   2> 2166650 INFO  (coreZkRegister-10755-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.ZkController Core needs to recover:collection1
   [junit4]   2> 2166651 INFO  (updateExecutor-9650-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
   [junit4]   2> 2166652 INFO  (recoveryExecutor-9651-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
   [junit4]   2> 2166652 INFO  (recoveryExecutor-9651-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
   [junit4]   2> 2166652 INFO  (recoveryExecutor-9651-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1]
   [junit4]   2> 2166652 INFO  (recoveryExecutor-9651-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
   [junit4]   2> 2166652 INFO  (recoveryExecutor-9651-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core [collection1] as recovering, leader is [http://127.0.0.1:38623/jd/pj/collection1/] and I am [http://127.0.0.1:53265/jd/pj/collection1/]
   [junit4]   2> 2166656 INFO  (recoveryExecutor-9651-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:38623/jd/pj]; [WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:53265_jd%252Fpj&coreNodeName=core_node3&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
   [junit4]   2> 2166657 INFO  (SocketProxy-Acceptor-38623) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=59443,localport=38623], receiveBufferSize:130880
   [junit4]   2> 2166664 INFO  (SocketProxy-Acceptor-38623) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=65023,localport=35685], receiveBufferSize=139264
   [junit4]   2> 2166666 INFO  (qtp632224398-43436) [n:127.0.0.1:38623_jd%2Fpj    ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
   [junit4]   2> 2166667 INFO  (qtp632224398-43436) [n:127.0.0.1:38623_jd%2Fpj    ] o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 (shard2 of collection1) have state: recovering
   [junit4]   2> 2166667 INFO  (qtp632224398-43436) [n:127.0.0.1:38623_jd%2Fpj    ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard2, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:53265_jd%2Fpj, coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: core_node3:{"core":"collection1","base_url":"http://127.0.0.1:53265/jd/pj","node_name":"127.0.0.1:53265_jd%2Fpj","state":"down"}
   [junit4]   2> 2166758 INFO  (zkCallback-9647-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 2166758 INFO  (zkCallback-9641-thread-2-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 2166758 INFO  (zkCallback-9653-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj) [n:127.0.0.1:53265_jd%2Fpj    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 2166979 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.SolrTestCaseJ4 ###Starting test
   [junit4]   2> 2166979 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Wait for recoveries to finish - wait 30000 for each attempt
   [junit4]   2> 2166979 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):30000
   [junit4]   2> 2167668 INFO  (qtp632224398-43436) [n:127.0.0.1:38623_jd%2Fpj    ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard2, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:53265_jd%2Fpj, coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: core_node3:{"core":"collection1","base_url":"http://127.0.0.1:53265/jd/pj","node_name":"127.0.0.1:53265_jd%2Fpj","state":"recovering"}
   [junit4]   2> 2167668 INFO  (qtp632224398-43436) [n:127.0.0.1:38623_jd%2Fpj    ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
   [junit4]   2> 2167669 INFO  (qtp632224398-43436) [n:127.0.0.1:38623_jd%2Fpj    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:53265_jd%252Fpj&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node3&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1002
   [junit4]   2> 2174669 INFO  (recoveryExecutor-9651-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:38623/jd/pj/collection1/] - recoveringAfterStartup=[true]
   [junit4]   2> 2174669 INFO  (recoveryExecutor-9651-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.PeerSync PeerSync: core=collection1 url=http://127.0.0.1:53265/jd/pj START replicas=[http://127.0.0.1:38623/jd/pj/collection1/] nUpdates=100
   [junit4]   2> 2174671 INFO  (SocketProxy-Acceptor-38623) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=34866,localport=38623], receiveBufferSize:130880
   [junit4]   2> 2174672 INFO  (SocketProxy-Acceptor-38623) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=65023,localport=64548], receiveBufferSize=139264
   [junit4]   2> 2174674 INFO  (qtp632224398-43441) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
   [junit4]   2> 2174674 INFO  (qtp632224398-43441) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.S.Request [collection1]  webapp=/jd/pj path=/get params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2174675 INFO  (recoveryExecutor-9651-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
   [junit4]   2> 2174675 INFO  (recoveryExecutor-9651-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.PeerSync We are already in sync. No need to do a PeerSync 
   [junit4]   2> 2174675 INFO  (recoveryExecutor-9651-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2174675 INFO  (recoveryExecutor-9651-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 2174675 INFO  (recoveryExecutor-9651-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 2174675 INFO  (recoveryExecutor-9651-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy PeerSync stage of recovery was successful.
   [junit4]   2> 2174675 INFO  (recoveryExecutor-9651-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Replaying updates buffered during PeerSync.
   [junit4]   2> 2174675 INFO  (recoveryExecutor-9651-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy No replay needed.
   [junit4]   2> 2174675 INFO  (recoveryExecutor-9651-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Registering as Active after recovery.
   [junit4]   2> 2174681 INFO  (zkCallback-9647-thread-1-processing-n:127.0.0.1:43728_jd%2Fpj) [n:127.0.0.1:43728_jd%2Fpj    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 2174681 INFO  (zkCallback-9641-thread-2-processing-n:127.0.0.1:38623_jd%2Fpj) [n:127.0.0.1:38623_jd%2Fpj    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 2174681 INFO  (zkCallback-9653-thread-1-processing-n:127.0.0.1:53265_jd%2Fpj) [n:127.0.0.1:53265_jd%2Fpj    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 2175007 INFO  (TEST-HttpPartitionTest.test-seed#[62308C8551BE9D75]) [    ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: collection1
   [junit4]   2> 2175007 INFO  (SocketProxy-Acceptor-44510) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=38472,localport=44510], receiveBufferSize:130880
   [junit4]   2> 2175009 INFO  (SocketProxy-Acceptor-44510) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=56111,localport=35491], receiveBufferSize=139264
   [junit4]   2> 2175013 INFO  (qtp1287113031-43398) [n:127.0.0.1:44510_jd%2Fpj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2175013 INFO  (qtp1287113031-43398) [n:127.0.0.1:44510_jd%2Fpj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 2175013 INFO  (qtp1287113031-43398) [n:127.0.0.1:44510_jd%2Fpj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 2175013 INFO  (qtp1287113031-43398) [n:127.0.0.1:44510_jd%2Fpj c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/jd/pj path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 0
   [junit4]   2> 2175016 INFO  (SocketProxy-Acceptor-43728) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=49747,localport=43728], receiveBufferSize:130880
   [junit4]   2> 2175017 INFO  (SocketProxy-Acceptor-43728) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=63879,localport=58042], receiveBufferSize=139264
   [junit4]   2> 2175052 INFO  (SocketProxy-Acceptor-43728) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=62032,localport=43728], receiveBufferSize:130880
   [junit4]   2> 2175053 INFO  (SocketProxy-Acceptor-43728) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=63879,localport=49428], receiveBufferSize=139264
   [junit4]   2> 2175056 INFO  (SocketProxy-Acceptor-53265) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=50582,localport=53265], receiveBufferSize:130880
   [junit4]   2> 2175056 INFO  (SocketProxy-Acceptor-38623) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=45759,localport=38623], receiveBufferSize:130880
   [junit4]   2> 2175056 INFO  (SocketProxy-Acceptor-53265) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=53634,localport=63960], receiveBufferSize=139264
   [junit4]   2> 2175057 INFO  (SocketProxy-Acceptor-38623) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=65023,localport=58936], receiveBufferSize=139264
   [junit4]   2> 2175059 INFO  (qtp1817267441-43468) [n:127.0.0.1:43728_jd%2Fpj c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2175060 INFO  (qtp1817267441-43468) [n:127.0.0.1:43728_jd%2Fpj c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 2175060 INFO  (qtp243989786-43496) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2175060 INFO  (qtp243989786-43496) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 2175060 INFO  (qtp1817267441-43468) [n:127.0.0.1:43728_jd%2Fpj c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 2175060 INFO  (qtp243989786-43496) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 2175060 INFO  (qtp632224398-43439) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2175060 INFO  (qtp632224398-43439) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 2175061 INFO  (qtp243989786-43496) [n:127.0.0.1:53265_jd%2Fpj c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/jd/pj path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:43728/jd/pj/collection1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 1
   [junit4]   2> 2175061 INFO  (qtp632224398-43439) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 2175061 INFO  (qtp632224398-43439) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/jd/pj path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:43728/jd/pj/collection1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 1
   [junit4]   2> 2175060 INFO  (qtp1817267441-43468) [n:127.0.0.1:43728_jd%2Fpj c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/jd/pj path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:43728/jd/pj/collection1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 0
   [junit4]   2> 2175061 INFO  (qtp1817267441-43469) [n:127.0.0.1:43728_jd%2Fpj c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/jd/pj path=/update params={_stateVer_=collection1:9&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 42
   [junit4]   2> 2175064 INFO  (SocketProxy-Acceptor-38623) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=60014,localport=38623], receiveBufferSize:130880
   [junit4]   2> 2175065 INFO  (SocketProxy-Acceptor-38623) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=65023,localport=42290], receiveBufferSize=139264
   [junit4]   2> 2175068 INFO  (qtp632224398-43436) [n:127.0.0.1:38623_jd%2Fpj c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.S.Request [collection1]  webapp=/jd/pj path=/select params={q=*:*&distrib=false&tests=checkShardConsistency&rows=0&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 2175069 INFO  (SocketProxy-Acceptor-53265) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=52215,localport=53265], receiveBufferSize:130880
   [junit4]   2> 2175069 INFO  (SocketProxy-Acceptor-53265) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=53634,localport=40284], 

[...truncated too long message...]

j.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1173)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1062)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1004)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
   [junit4]    > 	at org.apache.solr.cloud.HttpPartitionTest.sendDoc(HttpPartitionTest.java:609)
   [junit4]    > 	at org.apache.solr.cloud.HttpPartitionTest.sendDoc(HttpPartitionTest.java:595)
   [junit4]    > 	at org.apache.solr.cloud.HttpPartitionTest.testRf2(HttpPartitionTest.java:294)
   [junit4]    > 	at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:125)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:53265/jd/pj/c8n_1x2_shard1_replica1: Expected mime type application/octet-stream but got text/html. <html>
   [junit4]    > <head>
   [junit4]    > <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
   [junit4]    > <title>Error 404 </title>
   [junit4]    > </head>
   [junit4]    > <body>
   [junit4]    > <h2>HTTP ERROR: 404</h2>
   [junit4]    > <p>Problem accessing /jd/pj/c8n_1x2_shard1_replica1/update. Reason:
   [junit4]    > <pre>    Not Found</pre></p>
   [junit4]    > <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.8.v20160314</a><hr/>
   [junit4]    > </body>
   [junit4]    > </html>
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:561)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:262)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:435)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:387)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:742)
   [junit4]    > 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]    > 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    > 	... 1 more
   [junit4]   2> 2258549 INFO  (SUITE-HttpPartitionTest-seed#[62308C8551BE9D75]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_62308C8551BE9D75-001
   [junit4]   2> Oct 09, 2016 3:59:47 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62), sim=ClassicSimilarity, locale=ar-SD, timezone=Asia/Vientiane
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=3,threads=1,free=310439264,total=518979584
   [junit4]   2> NOTE: All tests run in this JVM: [DistanceFunctionTest, TermVectorComponentTest, TestRangeQuery, OverriddenZkACLAndCredentialsProvidersTest, TestUniqueKeyFieldResource, TestZkChroot, QueryParsingTest, UpdateRequestProcessorFactoryTest, SolrInfoMBeanTest, TestLMJelinekMercerSimilarityFactory, FieldAnalysisRequestHandlerTest, WordBreakSolrSpellCheckerTest, TestScoreJoinQPNoScore, SolrIndexSplitterTest, PreAnalyzedFieldTest, TestRandomCollapseQParserPlugin, TestValueSourceCache, CollectionTooManyReplicasTest, BlockDirectoryTest, TestQueryUtils, StatsComponentTest, TestFieldTypeResource, SpellingQueryConverterTest, TestDefaultSearchFieldResource, NoCacheHeaderTest, TestReload, TestMacroExpander, PeerSyncReplicationTest, SearchHandlerTest, ConvertedLegacyTest, DistributedTermsComponentTest, WrapperMergePolicyFactoryTest, SpellPossibilityIteratorTest, SolrCoreCheckLockOnStartupTest, ReplicationFactorTest, TestNamedUpdateProcessors, TestFieldCacheSort, DebugComponentTest, CurrencyFieldOpenExchangeTest, TestNumericTerms64, DistributedIntervalFacetingTest, DistribCursorPagingTest, BasicAuthIntegrationTest, EchoParamsTest, TestFieldResource, CoreAdminHandlerTest, TermVectorComponentDistributedTest, TestSolrQueryParser, MBeansHandlerTest, TestSizeLimitedDistributedMap, TestMinMaxOnMultiValuedField, TestLocalFSCloudBackupRestore, BlockJoinFacetSimpleTest, TestTrackingShardHandlerFactory, TestOmitPositions, DirectUpdateHandlerOptimizeTest, TestNRTOpen, DocExpirationUpdateProcessorFactoryTest, FileBasedSpellCheckerTest, TestMergePolicyConfig, TestSolr4Spatial2, SliceStateTest, TestTolerantUpdateProcessorRandomCloud, DeleteInactiveReplicaTest, TestSolr4Spatial, TestDFISimilarityFactory, TestRandomFaceting, TestQuerySenderListener, TriLevelCompositeIdRoutingTest, DateRangeFieldTest, TestSchemaManager, CSVRequestHandlerTest, TestFieldCacheReopen, QueryResultKeyTest, HLLUtilTest, CdcrBootstrapTest, TestDocBasedVersionConstraints, TestWriterPerf, TestStressLiveNodes, ResourceLoaderTest, TestJoin, TestLegacyFieldCache, RAMDirectoryFactoryTest, SolrCloudExampleTest, ReturnFieldsTest, FullSolrCloudDistribCmdsTest, NotRequiredUniqueKeyTest, HdfsBasicDistributedZk2Test, TestExactStatsCache, SortByFunctionTest, MergeStrategyTest, TestAtomicUpdateErrorCases, TestSuggestSpellingConverter, TestSimpleQParserPlugin, HdfsUnloadDistributedZkTest, TestSolrCloudWithKerberosAlt, TestLMDirichletSimilarityFactory, TestSolrDynamicMBean, TestSolrCLIRunExample, BadIndexSchemaTest, SignatureUpdateProcessorFactoryTest, HdfsRecoverLeaseTest, TestMiniSolrCloudCluster, CollectionsAPIDistributedZkTest, CleanupOldIndexTest, TestConfigSetsAPIZkFailure, TestMissingGroups, TestLuceneMatchVersion, IndexSchemaTest, MinimalSchemaTest, TestDynamicFieldResource, TestSSLRandomization, TestMiniSolrCloudClusterSSL, TestConfig, TestSubQueryTransformerCrossCore, TestFastWriter, TestQueryTypes, RegexBoostProcessorTest, FullHLLTest, HdfsRestartWhileUpdatingTest, TestReloadDeadlock, TestDistribIDF, TestHighFrequencyDictionaryFactory, CdcrVersionReplicationTest, HdfsChaosMonkeyNothingIsSafeTest, SolrIndexConfigTest, TestDistributedGrouping, BlockJoinFacetDistribTest, TestRealTimeGet, TestUseDocValuesAsStored2, TestExpandComponent, TestManagedSynonymFilterFactory, DistributedFacetPivotSmallTest, TestCloudPseudoReturnFields, HighlighterConfigTest, TestSchemaVersionResource, ShardRoutingCustomTest, TestConfigSetsAPI, ShowFileRequestHandlerTest, CopyFieldTest, TestInitQParser, TestDistributedMissingSort, TestCrossCoreJoin, OverseerModifyCollectionTest, UpdateParamsTest, TestSQLHandler, BinaryUpdateRequestHandlerTest, RequestHandlersTest, SegmentsInfoRequestHandlerTest, TestHdfsUpdateLog, TestSolrQueryParserDefaultOperatorResource, LeaderInitiatedRecoveryOnShardRestartTest, URLClassifyProcessorTest, BasicDistributedZkTest, BasicDistributedZk2Test, OpenCloseCoreStressTest, OverseerTest, BasicZkTest, TestReplicationHandler, AutoCommitTest, TestRecovery, TestHashPartitioner, DistributedSpellCheckComponentTest, TestStressReorder, SimpleFacetsTest, SolrCoreTest, SpellCheckComponentTest, TestGroupingSearch, SolrCmdDistributorTest, QueryElevationComponentTest, TestFiltering, TestFunctionQuery, BasicFunctionalityTest, TestBadConfig, TestIndexSearcher, HighlighterTest, SoftAutoCommitTest, DistributedQueryElevationComponentTest, TestCoreDiscovery, TestExtendedDismaxParser, SpellCheckCollatorTest, TestFoldingMultitermQuery, DocValuesTest, SuggesterTSTTest, SuggesterTest, SpatialFilterTest, TestCSVLoader, SchemaVersionSpecificBehaviorTest, FieldMutatingUpdateProcessorTest, DefaultValueUpdateProcessorTest, DocValuesMultiTest, XsltUpdateRequestHandlerTest, TestSurroundQueryParser, DisMaxRequestHandlerTest, StandardRequestHandlerTest, DirectSolrSpellCheckerTest, DocumentAnalysisRequestHandlerTest, TestIndexingPerformance, FastVectorHighlighterTest, LoggingHandlerTest, TestJmxIntegration, TestCSVResponseWriter, JsonLoaderTest, TestComponentsName, TestQuerySenderNoQuery, TestSolrDeletionPolicy2, TestDocSet, TestBinaryField, TestSearchPerf, TestElisionMultitermQuery, OutputWriterTest, TestFuzzyAnalyzedSuggestions, ExternalFileFieldSortTest, TestSolrCoreProperties, TestCodecSupport, SynonymTokenizerTest, TestSweetSpotSimilarityFactory, TestDFRSimilarityFactory, TestJmxMonitoredMap, TimeZoneUtilsTest, TestFastOutputStream, ScriptEngineTest, OpenExchangeRatesOrgProviderTest, TestFastLRUCache, DateMathParserTest, TestSystemIdResolver, PrimUtilsTest, DateFieldTest, ClusterStateTest, TestLRUCache, TestUtils, ZkNodePropsTest, UUIDFieldTest, CircularListTest, DistributedMLTComponentTest, TestRTGBase, CursorPagingTest, TestHighlightDedupGrouping, TestSimpleTrackingShardHandler, TestEmbeddedSolrServerConstructors, TestJettySolrRunner, ConnectionReuseTest, AliasIntegrationTest, AssignTest, AsyncCallRequestStatusResponseTest, CdcrReplicationDistributedZkTest, CdcrReplicationHandlerTest, CdcrRequestHandlerTest, CollectionReloadTest, CollectionStateFormat2Test, CollectionsAPISolrJTest, ConnectionManagerTest, CustomCollectionTest, DistributedVersionInfoTest, HttpPartitionTest]
   [junit4] Completed [464/637 (1!)] on J1 in 99.53s, 1 test, 1 error <<< FAILURES!

[...truncated 60930 lines...]


[JENKINS] Lucene-Solr-6.x-Solaris (64bit/jdk1.8.0) - Build # 439 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Solaris/439/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

All tests passed

Build Log:
[...truncated 53372 lines...]
-ecj-javadoc-lint-src:
    [mkdir] Created dir: /var/tmp/ecj2053804128
 [ecj-lint] Compiling 127 source files to /var/tmp/ecj2053804128
 [ecj-lint] ----------
 [ecj-lint] 1. ERROR in /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/lucene/benchmark/src/java/org/apache/lucene/benchmark/byTask/tasks/ReadTask.java (at line 21)
 [ecj-lint] 	import java.util.Collection;
 [ecj-lint] 	       ^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import java.util.Collection is never used
 [ecj-lint] ----------
 [ecj-lint] 2. ERROR in /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/lucene/benchmark/src/java/org/apache/lucene/benchmark/byTask/tasks/ReadTask.java (at line 22)
 [ecj-lint] 	import java.util.HashSet;
 [ecj-lint] 	       ^^^^^^^^^^^^^^^^^
 [ecj-lint] The import java.util.HashSet is never used
 [ecj-lint] ----------
 [ecj-lint] 3. ERROR in /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/lucene/benchmark/src/java/org/apache/lucene/benchmark/byTask/tasks/ReadTask.java (at line 24)
 [ecj-lint] 	import java.util.List;
 [ecj-lint] 	       ^^^^^^^^^^^^^^
 [ecj-lint] The import java.util.List is never used
 [ecj-lint] ----------
 [ecj-lint] 4. ERROR in /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/lucene/benchmark/src/java/org/apache/lucene/benchmark/byTask/tasks/ReadTask.java (at line 25)
 [ecj-lint] 	import java.util.Set;
 [ecj-lint] 	       ^^^^^^^^^^^^^
 [ecj-lint] The import java.util.Set is never used
 [ecj-lint] ----------
 [ecj-lint] 5. ERROR in /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/lucene/benchmark/src/java/org/apache/lucene/benchmark/byTask/tasks/ReadTask.java (at line 27)
 [ecj-lint] 	import org.apache.lucene.analysis.Analyzer;
 [ecj-lint] 	       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import org.apache.lucene.analysis.Analyzer is never used
 [ecj-lint] ----------
 [ecj-lint] 6. ERROR in /export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/lucene/benchmark/src/java/org/apache/lucene/benchmark/byTask/tasks/ReadTask.java (at line 33)
 [ecj-lint] 	import org.apache.lucene.index.IndexableField;
 [ecj-lint] 	       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import org.apache.lucene.index.IndexableField is never used
 [ecj-lint] ----------
 [ecj-lint] 6 problems (6 errors)

BUILD FAILED
/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/build.xml:765: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/build.xml:101: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/lucene/build.xml:204: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/lucene/common-build.xml:2177: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/lucene/common-build.xml:1992: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-6.x-Solaris/lucene/common-build.xml:2031: Compile failed; see the compiler error output for details.

Total time: 86 minutes 19 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any