You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2018/04/10 03:53:05 UTC

[JENKINS] Lucene-Solr-BadApples-master-Linux (32bit/jdk1.8.0_162) - Build # 20 - Unstable!

Error processing tokens: Error while parsing action 'Text/ZeroOrMore/FirstOf/Token/DelimitedToken/DelimitedToken_Action3' at input position (line 79, pos 4):
)"}
   ^

java.lang.OutOfMemoryError: Java heap space

[JENKINS] Lucene-Solr-BadApples-master-Linux (64bit/jdk1.8.0_162) - Build # 24 - Failure!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-BadApples-master-Linux/24/
Java: 64bit/jdk1.8.0_162 -XX:+UseCompressedOops -XX:+UseParallelGC

All tests passed

Build Log:
[...truncated 61965 lines...]
-ecj-javadoc-lint-tests:
    [mkdir] Created dir: /tmp/ecj1026895660
 [ecj-lint] Compiling 896 source files to /tmp/ecj1026895660
 [ecj-lint] invalid Class-Path header in manifest of jar file: /home/jenkins/.ivy2/cache/org.restlet.jee/org.restlet/jars/org.restlet-2.3.0.jar
 [ecj-lint] invalid Class-Path header in manifest of jar file: /home/jenkins/.ivy2/cache/org.restlet.jee/org.restlet.ext.servlet/jars/org.restlet.ext.servlet-2.3.0.jar
 [ecj-lint] ----------
 [ecj-lint] 1. WARNING in /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/core/src/test/org/apache/solr/analysis/TokenizerChainTest.java (at line 37)
 [ecj-lint] 	TokenizerChain tokenizerChain = new TokenizerChain(
 [ecj-lint] 	               ^^^^^^^^^^^^^^
 [ecj-lint] Resource leak: 'tokenizerChain' is never closed
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 2. ERROR in /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/core/src/test/org/apache/solr/cloud/DeleteReplicaTest.java (at line 31)
 [ecj-lint] 	import java.util.function.Supplier;
 [ecj-lint] 	       ^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import java.util.function.Supplier is never used
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 3. ERROR in /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/core/src/test/org/apache/solr/cloud/ForceLeaderTest.java (at line 25)
 [ecj-lint] 	import java.util.concurrent.TimeUnit;
 [ecj-lint] 	       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import java.util.concurrent.TimeUnit is never used
 [ecj-lint] ----------
 [ecj-lint] 4. ERROR in /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/core/src/test/org/apache/solr/cloud/ForceLeaderTest.java (at line 26)
 [ecj-lint] 	import java.util.stream.Collectors;
 [ecj-lint] 	       ^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import java.util.stream.Collectors is never used
 [ecj-lint] ----------
 [ecj-lint] 5. ERROR in /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/core/src/test/org/apache/solr/cloud/ForceLeaderTest.java (at line 32)
 [ecj-lint] 	import org.apache.solr.cloud.overseer.OverseerAction;
 [ecj-lint] 	       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import org.apache.solr.cloud.overseer.OverseerAction is never used
 [ecj-lint] ----------
 [ecj-lint] 6. ERROR in /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/core/src/test/org/apache/solr/cloud/ForceLeaderTest.java (at line 38)
 [ecj-lint] 	import org.apache.solr.common.cloud.ZkNodeProps;
 [ecj-lint] 	       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import org.apache.solr.common.cloud.ZkNodeProps is never used
 [ecj-lint] ----------
 [ecj-lint] 7. ERROR in /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/core/src/test/org/apache/solr/cloud/ForceLeaderTest.java (at line 39)
 [ecj-lint] 	import org.apache.solr.common.cloud.ZkStateReader;
 [ecj-lint] 	       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import org.apache.solr.common.cloud.ZkStateReader is never used
 [ecj-lint] ----------
 [ecj-lint] 8. ERROR in /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/core/src/test/org/apache/solr/cloud/ForceLeaderTest.java (at line 41)
 [ecj-lint] 	import org.apache.solr.common.util.Utils;
 [ecj-lint] 	       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import org.apache.solr.common.util.Utils is never used
 [ecj-lint] ----------
 [ecj-lint] ----------
 [ecj-lint] 9. ERROR in /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/core/src/test/org/apache/solr/cloud/MoveReplicaTest.java (at line 25)
 [ecj-lint] 	import java.util.HashSet;
 [ecj-lint] 	       ^^^^^^^^^^^^^^^^^
 [ecj-lint] The import java.util.HashSet is never used
 [ecj-lint] ----------
 [ecj-lint] 10. ERROR in /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/core/src/test/org/apache/solr/cloud/MoveReplicaTest.java (at line 42)
 [ecj-lint] 	import org.apache.solr.common.cloud.CollectionStateWatcher;
 [ecj-lint] 	       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import org.apache.solr.common.cloud.CollectionStateWatcher is never used
 [ecj-lint] ----------
 [ecj-lint] 11. ERROR in /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/core/src/test/org/apache/solr/cloud/MoveReplicaTest.java (at line 46)
 [ecj-lint] 	import org.apache.solr.common.cloud.ZkStateReaderAccessor;
 [ecj-lint] 	       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 [ecj-lint] The import org.apache.solr.common.cloud.ZkStateReaderAccessor is never used
 [ecj-lint] ----------
 [ecj-lint] 11 problems (10 errors, 1 warning)

BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/build.xml:642: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/build.xml:101: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build.xml:690: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/lucene/common-build.xml:2095: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/lucene/common-build.xml:2128: Compile failed; see the compiler error output for details.

Total time: 74 minutes 31 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2

[JENKINS] Lucene-Solr-BadApples-master-Linux (64bit/jdk-10) - Build # 23 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-BadApples-master-Linux/23/
Java: 64bit/jdk-10 -XX:-UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove

Error Message:
No live SolrServers available to handle this request:[https://127.0.0.1:37329/solr/MoveReplicaHDFSTest_failed_coll_true, https://127.0.0.1:38709/solr/MoveReplicaHDFSTest_failed_coll_true]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:37329/solr/MoveReplicaHDFSTest_failed_coll_true, https://127.0.0.1:38709/solr/MoveReplicaHDFSTest_failed_coll_true]
	at __randomizedtesting.SeedInfo.seed([1C49CA1791E3FFDD:B68419E526302A0D]:0)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:462)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:993)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
	at org.apache.solr.cloud.MoveReplicaTest.testFailedMove(MoveReplicaTest.java:310)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:38709/solr/MoveReplicaHDFSTest_failed_coll_true: no servers hosting shard: shard1
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:436)
	... 45 more




Build Log:
[...truncated 13806 lines...]
   [junit4] Suite: org.apache.solr.cloud.MoveReplicaHDFSTest
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_1C49CA1791E3FFDD-001/init-core-data-001
   [junit4]   2> 1781179 WARN  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=7 numCloses=7
   [junit4]   2> 1781180 INFO  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 1781180 INFO  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0, clientAuth=0.0/0.0)
   [junit4]   2> 1781180 INFO  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1781180 INFO  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_1C49CA1791E3FFDD-001/tempDir-001
   [junit4]   2> 1781181 INFO  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1781181 INFO  (Thread-1139) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1781181 INFO  (Thread-1139) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1781182 ERROR (Thread-1139) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1781281 INFO  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:34475
   [junit4]   2> 1781283 INFO  (zkConnectionManagerCallback-1264-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1781286 INFO  (jetty-launcher-1261-thread-3) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T15:27:37-06:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1781286 INFO  (jetty-launcher-1261-thread-1) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T15:27:37-06:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1781286 INFO  (jetty-launcher-1261-thread-2) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T15:27:37-06:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1781286 INFO  (jetty-launcher-1261-thread-4) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T15:27:37-06:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1781290 INFO  (jetty-launcher-1261-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1781290 INFO  (jetty-launcher-1261-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1781290 INFO  (jetty-launcher-1261-thread-4) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1781290 INFO  (jetty-launcher-1261-thread-2) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 1781290 INFO  (jetty-launcher-1261-thread-4) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1781290 INFO  (jetty-launcher-1261-thread-4) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 1781290 INFO  (jetty-launcher-1261-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@57124143{/solr,null,AVAILABLE}
   [junit4]   2> 1781290 INFO  (jetty-launcher-1261-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3104f24a{/solr,null,AVAILABLE}
   [junit4]   2> 1781291 INFO  (jetty-launcher-1261-thread-3) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1781291 INFO  (jetty-launcher-1261-thread-3) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1781291 INFO  (jetty-launcher-1261-thread-3) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 1781291 INFO  (jetty-launcher-1261-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1781291 INFO  (jetty-launcher-1261-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1781291 INFO  (jetty-launcher-1261-thread-1) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 1781292 INFO  (jetty-launcher-1261-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5b6d46af{/solr,null,AVAILABLE}
   [junit4]   2> 1781292 INFO  (jetty-launcher-1261-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@44a7384e{/solr,null,AVAILABLE}
   [junit4]   2> 1781292 INFO  (jetty-launcher-1261-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@190f2171{SSL,[ssl, http/1.1]}{127.0.0.1:37329}
   [junit4]   2> 1781292 INFO  (jetty-launcher-1261-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5aa1fe39{SSL,[ssl, http/1.1]}{127.0.0.1:38863}
   [junit4]   2> 1781292 INFO  (jetty-launcher-1261-thread-1) [    ] o.e.j.s.Server Started @1781337ms
   [junit4]   2> 1781292 INFO  (jetty-launcher-1261-thread-3) [    ] o.e.j.s.Server Started @1781337ms
   [junit4]   2> 1781292 INFO  (jetty-launcher-1261-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37329}
   [junit4]   2> 1781292 INFO  (jetty-launcher-1261-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38863}
   [junit4]   2> 1781292 ERROR (jetty-launcher-1261-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1781292 ERROR (jetty-launcher-1261-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1781292 INFO  (jetty-launcher-1261-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1781292 INFO  (jetty-launcher-1261-thread-3) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1781292 INFO  (jetty-launcher-1261-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1781292 INFO  (jetty-launcher-1261-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1781292 INFO  (jetty-launcher-1261-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1781292 INFO  (jetty-launcher-1261-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1781292 INFO  (jetty-launcher-1261-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1781292 INFO  (jetty-launcher-1261-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1781292 INFO  (jetty-launcher-1261-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-04-16T03:34:20.196761Z
   [junit4]   2> 1781292 INFO  (jetty-launcher-1261-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-04-16T03:34:20.196776Z
   [junit4]   2> 1781293 INFO  (jetty-launcher-1261-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4b9d049b{SSL,[ssl, http/1.1]}{127.0.0.1:34547}
   [junit4]   2> 1781293 INFO  (zkConnectionManagerCallback-1268-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1781293 INFO  (jetty-launcher-1261-thread-2) [    ] o.e.j.s.Server Started @1781339ms
   [junit4]   2> 1781293 INFO  (jetty-launcher-1261-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@66b0e289{SSL,[ssl, http/1.1]}{127.0.0.1:45095}
   [junit4]   2> 1781293 INFO  (zkConnectionManagerCallback-1267-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1781293 INFO  (jetty-launcher-1261-thread-4) [    ] o.e.j.s.Server Started @1781339ms
   [junit4]   2> 1781293 INFO  (jetty-launcher-1261-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34547}
   [junit4]   2> 1781293 INFO  (jetty-launcher-1261-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45095}
   [junit4]   2> 1781294 INFO  (jetty-launcher-1261-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1781294 INFO  (jetty-launcher-1261-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1781294 ERROR (jetty-launcher-1261-thread-4) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1781294 ERROR (jetty-launcher-1261-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1781294 INFO  (jetty-launcher-1261-thread-4) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1781294 INFO  (jetty-launcher-1261-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1781294 INFO  (jetty-launcher-1261-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1781294 INFO  (jetty-launcher-1261-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1781294 INFO  (jetty-launcher-1261-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1781294 INFO  (jetty-launcher-1261-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1781294 INFO  (jetty-launcher-1261-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1781294 INFO  (jetty-launcher-1261-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1781294 INFO  (jetty-launcher-1261-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-04-16T03:34:20.198295Z
   [junit4]   2> 1781294 INFO  (jetty-launcher-1261-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-04-16T03:34:20.198303Z
   [junit4]   2> 1781299 INFO  (zkConnectionManagerCallback-1271-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1781299 INFO  (jetty-launcher-1261-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1781300 INFO  (zkConnectionManagerCallback-1272-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1781300 INFO  (jetty-launcher-1261-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1781300 INFO  (jetty-launcher-1261-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34475/solr
   [junit4]   2> 1781300 INFO  (jetty-launcher-1261-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34475/solr
   [junit4]   2> 1781302 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100517f1f340004, likely client has closed socket
   [junit4]   2> 1781303 INFO  (zkConnectionManagerCallback-1279-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1781303 INFO  (zkConnectionManagerCallback-1280-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1781303 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100517f1f340005, likely client has closed socket
   [junit4]   2> 1781303 DEBUG (jetty-launcher-1261-thread-1) [n:127.0.0.1:37329_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$385/682972725@697b547c
   [junit4]   2> 1781303 DEBUG (jetty-launcher-1261-thread-3) [n:127.0.0.1:38863_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$385/682972725@630b287d
   [junit4]   2> 1781304 INFO  (jetty-launcher-1261-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34475/solr
   [junit4]   2> 1781304 INFO  (zkConnectionManagerCallback-1284-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1781306 INFO  (jetty-launcher-1261-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34475/solr
   [junit4]   2> 1781309 INFO  (zkConnectionManagerCallback-1289-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1781309 INFO  (zkConnectionManagerCallback-1287-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1781309 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100517f1f340008, likely client has closed socket
   [junit4]   2> 1781309 INFO  (zkConnectionManagerCallback-1292-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1781309 DEBUG (jetty-launcher-1261-thread-2) [n:127.0.0.1:34547_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$385/682972725@a6b5f9d
   [junit4]   2> 1781310 DEBUG (jetty-launcher-1261-thread-4) [n:127.0.0.1:45095_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$385/682972725@4961aad3
   [junit4]   2> 1781312 INFO  (zkConnectionManagerCallback-1296-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1781312 INFO  (zkConnectionManagerCallback-1298-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1781317 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 1781317 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1781321 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 1781321 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1781322 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 1781322 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 1781322 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
   [junit4]   2> 1781322 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1781322 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 1781322 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1781323 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 1781324 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
   [junit4]   2> 1781324 INFO  (jetty-launcher-1261-thread-2) [n:127.0.0.1:34547_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1781324 DEBUG (jetty-launcher-1261-thread-2) [n:127.0.0.1:34547_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/72147200464125963-127.0.0.1:34547_solr-n_0000000000
   [junit4]   2> 1781325 INFO  (jetty-launcher-1261-thread-2) [n:127.0.0.1:34547_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34547_solr
   [junit4]   2> 1781325 INFO  (jetty-launcher-1261-thread-2) [n:127.0.0.1:34547_solr    ] o.a.s.c.Overseer Overseer (id=72147200464125963-127.0.0.1:34547_solr-n_0000000000) starting
   [junit4]   2> 1781326 INFO  (jetty-launcher-1261-thread-4) [n:127.0.0.1:45095_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1781326 DEBUG (jetty-launcher-1261-thread-4) [n:127.0.0.1:45095_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/72147200464125964-127.0.0.1:45095_solr-n_0000000001
   [junit4]   2> 1781326 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/overseer_elect/election/72147200464125963-127.0.0.1:34547_solr-n_0000000000
   [junit4]   2> 1781327 DEBUG (jetty-launcher-1261-thread-4) [n:127.0.0.1:45095_solr    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/72147200464125963-127.0.0.1:34547_solr-n_0000000000 to know if I could be the leader
   [junit4]   2> 1781327 INFO  (jetty-launcher-1261-thread-4) [n:127.0.0.1:45095_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45095_solr
   [junit4]   2> 1781327 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1781327 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1781333 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1781345 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1781347 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 1781347 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1781347 INFO  (jetty-launcher-1261-thread-1) [n:127.0.0.1:37329_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1781347 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 1781347 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1781347 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 1781347 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
   [junit4]   2> 1781348 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 1781348 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1781348 INFO  (jetty-launcher-1261-thread-3) [n:127.0.0.1:38863_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1781348 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 1781348 INFO  (jetty-launcher-1261-thread-1) [n:127.0.0.1:37329_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1781348 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1781348 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 1781348 DEBUG (jetty-launcher-1261-thread-1) [n:127.0.0.1:37329_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/72147200464125959-127.0.0.1:37329_solr-n_0000000002
   [junit4]   2> 1781348 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/overseer_elect/election/72147200464125964-127.0.0.1:45095_solr-n_0000000001
   [junit4]   2> 1781348 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
   [junit4]   2> 1781348 DEBUG (jetty-launcher-1261-thread-1) [n:127.0.0.1:37329_solr    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/72147200464125964-127.0.0.1:45095_solr-n_0000000001 to know if I could be the leader
   [junit4]   2> 1781348 DEBUG (jetty-launcher-1261-thread-1) [n:127.0.0.1:37329_solr    ] o.a.s.c.ZkController Publish node=127.0.0.1:37329_solr as DOWN
   [junit4]   2> 1781349 INFO  (jetty-launcher-1261-thread-3) [n:127.0.0.1:38863_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1781349 DEBUG (jetty-launcher-1261-thread-3) [n:127.0.0.1:38863_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/72147200464125961-127.0.0.1:38863_solr-n_0000000003
   [junit4]   2> 1781349 INFO  (jetty-launcher-1261-thread-1) [n:127.0.0.1:37329_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1781349 INFO  (jetty-launcher-1261-thread-1) [n:127.0.0.1:37329_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37329_solr
   [junit4]   2> 1781349 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/overseer_elect/election/72147200464125959-127.0.0.1:37329_solr-n_0000000002
   [junit4]   2> 1781349 DEBUG (jetty-launcher-1261-thread-3) [n:127.0.0.1:38863_solr    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/72147200464125959-127.0.0.1:37329_solr-n_0000000002 to know if I could be the leader
   [junit4]   2> 1781349 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1781349 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1781349 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1781349 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1781349 DEBUG (jetty-launcher-1261-thread-3) [n:127.0.0.1:38863_solr    ] o.a.s.c.ZkController Publish node=127.0.0.1:38863_solr as DOWN
   [junit4]   2> 1781352 INFO  (zkConnectionManagerCallback-1309-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1781352 INFO  (zkCallback-1297-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1781352 INFO  (zkCallback-1295-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1781352 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1781353 INFO  (zkCallback-1283-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1781353 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1781353 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1781353 INFO  (zkCallback-1297-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1781353 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1781353 INFO  (zkCallback-1295-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1781353 INFO  (zkCallback-1285-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1781353 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 1781353 INFO  (jetty-launcher-1261-thread-3) [n:127.0.0.1:38863_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1781353 INFO  (jetty-launcher-1261-thread-3) [n:127.0.0.1:38863_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38863_solr
   [junit4]   2> 1781353 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1781353 INFO  (jetty-launcher-1261-thread-2) [n:127.0.0.1:34547_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1781353 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 1781353 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1781353 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1781353 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1781353 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1781353 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1781353 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1781353 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1781353 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1781353 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1781353 INFO  (zkCallback-1283-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1781353 INFO  (zkCallback-1297-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1781353 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1781353 INFO  (zkCallback-1295-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1781353 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1781353 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 1781354 INFO  (zkCallback-1285-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1781354 INFO  (jetty-launcher-1261-thread-2) [n:127.0.0.1:34547_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34475/solr ready
   [junit4]   2> 1781354 INFO  (zkCallback-1308-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1781354 DEBUG (OverseerStateUpdate-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1781354 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue-work
   [junit4]   2> 1781355 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 1781355 DEBUG (OverseerStateUpdate-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:37329_solr"} current state version: 0
   [junit4]   2> 1781355 DEBUG (OverseerStateUpdate-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:37329_solr
   [junit4]   2> 1781355 DEBUG (OverseerStateUpdate-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:38863_solr"} current state version: 0
   [junit4]   2> 1781355 DEBUG (OverseerStateUpdate-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:38863_solr
   [junit4]   2> 1781360 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1781360 DEBUG (jetty-launcher-1261-thread-2) [n:127.0.0.1:34547_solr    ] o.a.s.c.ZkController Publish node=127.0.0.1:34547_solr as DOWN
   [junit4]   2> 1781361 DEBUG (OverseerAutoScalingTriggerThread-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1781361 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
   [junit4]   2> 1781361 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
   [junit4]   2> 1781361 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
   [junit4]   2> 1781361 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
   [junit4]   2> 1781361 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
   [junit4]   2> 1781361 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/autoscaling.json
   [junit4]   2> 1781361 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
   [junit4]   2> 1781361 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
   [junit4]   2> 1781361 DEBUG (OverseerAutoScalingTriggerThread-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 1781361 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
   [junit4]   2> 1781361 INFO  (jetty-launcher-1261-thread-2) [n:127.0.0.1:34547_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1781361 INFO  (jetty-launcher-1261-thread-2) [n:127.0.0.1:34547_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34547_solr
   [junit4]   2> 1781361 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 1781361 DEBUG (OverseerAutoScalingTriggerThread-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 1781361 DEBUG (OverseerAutoScalingTriggerThread-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 1781361 DEBUG (OverseerStateUpdate-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:34547_solr"} current state version: 0
   [junit4]   2> 1781361 DEBUG (OverseerStateUpdate-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:34547_solr
   [junit4]   2> 1781361 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1781361 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1781361 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1781361 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1781361 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1781361 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1781362 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1781362 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1781362 INFO  (zkCallback-1308-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1781362 INFO  (zkCallback-1295-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1781362 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1781362 INFO  (zkCallback-1297-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1781362 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1781362 INFO  (zkCallback-1285-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1781365 INFO  (zkCallback-1283-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1781365 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/autoscaling/events/.scheduled_maintenance
   [junit4]   2> 1781366 DEBUG (OverseerAutoScalingTriggerThread-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:38863_solr, 127.0.0.1:45095_solr, 127.0.0.1:34547_solr, 127.0.0.1:37329_solr]
   [junit4]   2> 1781371 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 1781371 DEBUG (ScheduledTrigger-2104-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1781372 DEBUG (OverseerAutoScalingTriggerThread-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1781372 DEBUG (OverseerAutoScalingTriggerThread-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 1781428 INFO  (jetty-launcher-1261-thread-1) [n:127.0.0.1:37329_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37329.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3917c857
   [junit4]   2> 1781434 INFO  (jetty-launcher-1261-thread-1) [n:127.0.0.1:37329_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37329.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3917c857
   [junit4]   2> 1781434 INFO  (jetty-launcher-1261-thread-1) [n:127.0.0.1:37329_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37329.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3917c857
   [junit4]   2> 1781439 INFO  (jetty-launcher-1261-thread-1) [n:127.0.0.1:37329_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_1C49CA1791E3FFDD-001/tempDir-001/node1/.
   [junit4]   2> 1781443 INFO  (zkConnectionManagerCallback-1314-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1781443 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 1781443 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1781443 INFO  (jetty-launcher-1261-thread-1) [n:127.0.0.1:37329_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1781443 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 1781443 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1781443 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 1781444 INFO  (jetty-launcher-1261-thread-1) [n:127.0.0.1:37329_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34475/solr ready
   [junit4]   2> 1781452 INFO  (jetty-launcher-1261-thread-2) [n:127.0.0.1:34547_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34547.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3917c857
   [junit4]   2> 1781457 INFO  (jetty-launcher-1261-thread-2) [n:127.0.0.1:34547_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34547.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3917c857
   [junit4]   2> 1781457 INFO  (jetty-launcher-1261-thread-2) [n:127.0.0.1:34547_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34547.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3917c857
   [junit4]   2> 1781457 INFO  (jetty-launcher-1261-thread-2) [n:127.0.0.1:34547_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_1C49CA1791E3FFDD-001/tempDir-001/node2/.
   [junit4]   2> 1781462 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
   [junit4]   2> 1781462 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 1781477 INFO  (jetty-launcher-1261-thread-4) [n:127.0.0.1:45095_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45095.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3917c857
   [junit4]   2> 1781480 INFO  (jetty-launcher-1261-thread-3) [n:127.0.0.1:38863_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38863.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3917c857
   [junit4]   2> 1781482 INFO  (jetty-launcher-1261-thread-4) [n:127.0.0.1:45095_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45095.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3917c857
   [junit4]   2> 1781482 INFO  (jetty-launcher-1261-thread-4) [n:127.0.0.1:45095_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45095.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3917c857
   [junit4]   2> 1781482 INFO  (jetty-launcher-1261-thread-4) [n:127.0.0.1:45095_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_1C49CA1791E3FFDD-001/tempDir-001/node4/.
   [junit4]   2> 1781484 INFO  (zkConnectionManagerCallback-1319-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1781485 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 1781485 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1781485 INFO  (jetty-launcher-1261-thread-4) [n:127.0.0.1:45095_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1781485 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 1781485 INFO  (jetty-launcher-1261-thread-3) [n:127.0.0.1:38863_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38863.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3917c857
   [junit4]   2> 1781485 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1781485 INFO  (jetty-launcher-1261-thread-3) [n:127.0.0.1:38863_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38863.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3917c857
   [junit4]   2> 1781485 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 1781485 INFO  (jetty-launcher-1261-thread-4) [n:127.0.0.1:45095_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34475/solr ready
   [junit4]   2> 1781486 INFO  (jetty-launcher-1261-thread-3) [n:127.0.0.1:38863_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_1C49CA1791E3FFDD-001/tempDir-001/node3/.
   [junit4]   2> 1781487 INFO  (zkConnectionManagerCallback-1324-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1781488 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 1781488 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1781488 INFO  (jetty-launcher-1261-thread-3) [n:127.0.0.1:38863_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1781488 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 1781488 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1781488 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 1781488 INFO  (jetty-launcher-1261-thread-3) [n:127.0.0.1:38863_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34475/solr ready
   [junit4]   2> 1781524 INFO  (zkConnectionManagerCallback-1326-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1781526 INFO  (zkConnectionManagerCallback-1331-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1781527 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 1781527 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1781527 INFO  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1781528 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 1781528 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1781528 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 1781528 INFO  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34475/solr ready
   [junit4]   2> 1781537 INFO  (qtp459697922-11167) [n:127.0.0.1:45095_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1781538 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/collection-queue-work
   [junit4]   2> 1781538 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/overseer/collection-queue-work/qnr-0000000000
   [junit4]   2> 1781538 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
   [junit4]   2> 1781538 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.ZkDistributedQueue Found child node with improper name: qnr-0000000000
   [junit4]   2> 1781538 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/collection-queue-work
   [junit4]   2> 1781539 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
   [junit4]   2> 1781539 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.ZkDistributedQueue Found child node with improper name: qnr-0000000000
   [junit4]   2> 1781539 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Returning topN elements: [/overseer/collection-queue-work/qn-0000000000, ]
   [junit4]   2> 1781539 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskProcessor Got 1 tasks from work-queue : [[org.apache.solr.cloud.OverseerTaskQueue$QueueEvent@2e684ecf]]
   [junit4]   2> 1781539 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskProcessor Marked task [/overseer/collection-queue-work/qn-0000000000] as running
   [junit4]   2> 1781539 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskProcessor Overseer Collection Message Handler: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{"operation":"overseerstatus"}
   [junit4]   2> 1781539 DEBUG (OverseerThreadFactory-2106-thread-1) [    ] o.a.s.c.OverseerTaskProcessor Runner processing /overseer/collection-queue-work/qn-0000000000
   [junit4]   2> 1781539 DEBUG (OverseerThreadFactory-2106-thread-1) [    ] o.a.s.c.a.c.OverseerCollectionMessageHandler OverseerCollectionMessageHandler.processMessage : overseerstatus , {"operation":"overseerstatus"}
   [junit4]   2> 1781539 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 1
   [junit4]   2> 1781539 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskProcessor RunningTasks: [/overseer/collection-queue-work/qn-0000000000]
   [junit4]   2> 1781539 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
   [junit4]   2> 1781539 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
   [junit4]   2> 1781539 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: [/overseer/collection-queue-work/qn-0000000000]
   [junit4]   2> 1781539 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Peeking for top 99 elements. ExcludeSet: [/overseer/collection-queue-work/qn-0000000000],[]
   [junit4]   2> 1781540 DEBUG (OverseerThreadFactory-2106-thread-1) [    ] o.a.s.c.OverseerTaskProcessor Completed task:[/overseer/collection-queue-work/qn-0000000000]
   [junit4]   2> 1781541 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: data: /solr/overseer/collection-queue-work/qnr-0000000000
   [junit4]   2> 1781541 DEBUG (zkCallback-1297-thread-2) [    ] o.a.s.c.OverseerTaskQueue NodeDataChanged fired on path /overseer/collection-queue-work/qnr-0000000000 state SyncConnected latchEventType null
   [junit4]   2> 1781541 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/collection-queue-work
   [junit4]   2> 1781541 DEBUG (OverseerThreadFactory-2106-thread-1) [    ] o.a.s.c.OverseerTaskProcessor Marked task [/overseer/collection-queue-work/qn-0000000000] as completed.
   [junit4]   2> 1781541 DEBUG (OverseerThreadFactory-2106-thread-1) [    ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
   [junit4]   2> 1781541 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
   [junit4]   2> 1781541 DEBUG (OverseerThreadFactory-2106-thread-1) [    ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
   [junit4]   2> 1781541 DEBUG (OverseerThreadFactory-2106-thread-1) [    ] o.a.s.c.OverseerTaskProcessor CompletedTasks: [/overseer/collection-queue-work/qn-0000000000]
   [junit4]   2> 1781541 DEBUG (OverseerThreadFactory-2106-thread-1) [    ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: [/overseer/collection-queue-work/qn-0000000000]
   [junit4]   2> 1781541 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.ZkDistributedQueue Found child node with improper name: qnr-0000000000
   [junit4]   2> 1781541 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/collection-queue-work
   [junit4]   2> 1781541 DEBUG (OverseerThreadFactory-2106-thread-1) [    ] o.a.s.c.OverseerTaskProcessor Overseer Collection Message Handler: Message id:/overseer/collection-queue-work/qn-0000000000 complete, response:{leader=127.0.0.1:34547_solr,overseer_queue_size=0,overseer_work_queue_size=0,overseer_collection_queue_size=2,overseer_operations={am_i_leader={requests=3,errors=0,avgRequestsPerSecond=16.149321101650916,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=0.12567966666666666,medianRequestTime=0.090236,75thPcRequestTime=0.228124,95thPcRequestTime=0.228124,99thPcRequestTime=0.228124,999thPcRequestTime=0.228124},downnode={requests=3,errors=0,avgRequestsPerSecond=16.21599743173195,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=0.020116,medianRequestTime=0.023522,75thPcRequestTime=0.023812,95thPcRequestTime=0.023812,99thPcRequestTime=0.023812,999thPcRequestTime=0.023812}},collection_operations={am_i_leader={requests=3,errors=0,avgRequestsPerSecond=16.717910498804883,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=0.11931666666666665,medianRequestTime=0.092595,75thPcRequestTime=0.191526,95thPcRequestTime=0.191526,99thPcRequestTime=0.191526,999thPcRequestTime=0.191526},overseerstatus={requests=0,errors=0,avgRequestsPerSecond=0.0,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=0.0,medianRequestTime=0.0,75thPcRequestTime=0.0,95thPcRequestTime=0.0,99thPcRequestTime=0.0,999thPcRequestTime=0.0}},overseer_queue={},overseer_internal_queue={peek={avgRequestsPerSecond=5.3957700184686415,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=0.074483,medianRequestTime=0.074483,75thPcRequestTime=0.074483,95thPcRequestTime=0.074483,99thPcRequestTime=0.074483,999thPcRequestTime=0.074483}},collection_queue={peektopn_wait2000={avgRequestsPerSecond=5.591976721271947,5minRateRequestsPerSecond=0.0,15minRateRequestsPerSecond=0.0,avgTimePerRequest=177.8948,medianRequestTime=177.8948,75thPcRequestTime=177.8948,95thPcRequestTime=177.8948,99thPcRequestTime=177.8948,999thPcRequestTime=177.8948}}}
   [junit4]   2> 1781541 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
   [junit4]   2> 1781541 INFO  (qtp459697922-11167) [n:127.0.0.1:45095_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=3
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 1781626 WARN  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 1781632 WARN  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1781633 INFO  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 1781635 INFO  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_localdomain_46003_hdfs____.uplm0l/webapp
   [junit4]   2> 1781706 INFO  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:46003
   [junit4]   2> 1781760 WARN  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1781761 INFO  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 1781764 INFO  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_36577_datanode____.rqflaf/webapp
   [junit4]   2> 1781827 INFO  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36577
   [junit4]   2> 1781850 WARN  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1781851 INFO  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 1781854 INFO  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_46309_datanode____.6e7rdr/webapp
   [junit4]   2> 1781917 ERROR (DataNode: [[[DISK]file:/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_1C49CA1791E3FFDD-001/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_1C49CA1791E3FFDD-001/tempDir-002/hdfsBaseDir/data/data2/]]  heartbeating to localhost.localdomain/127.0.0.1:35827) [    ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
   [junit4]   2> 1781922 INFO  (Block report processor) [    ] BlockStateChange BLOCK* processReport 0x4db8a25fd31cd: from storage DS-fb1531c8-c44a-4037-bcf7-9a83124da2b8 node DatanodeRegistration(127.0.0.1:42751, datanodeUuid=28cdadbc-ad1f-4d89-9e30-a8dc5075f1a0, infoPort=35123, infoSecurePort=0, ipcPort=46019, storageInfo=lv=-56;cid=testClusterID;nsid=1784438414;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 1781922 INFO  (Block report processor) [    ] BlockStateChange BLOCK* processReport 0x4db8a25fd31cd: from storage DS-523c2473-0368-4695-a463-22e4220d7b38 node DatanodeRegistration(127.0.0.1:42751, datanodeUuid=28cdadbc-ad1f-4d89-9e30-a8dc5075f1a0, infoPort=35123, infoSecurePort=0, ipcPort=46019, storageInfo=lv=-56;cid=testClusterID;nsid=1784438414;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 1781925 INFO  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46309
   [junit4]   2> 1782012 ERROR (DataNode: [[[DISK]file:/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_1C49CA1791E3FFDD-001/tempDir-002/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_1C49CA1791E3FFDD-001/tempDir-002/hdfsBaseDir/data/data4/]]  heartbeating to localhost.localdomain/127.0.0.1:35827) [    ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
   [junit4]   2> 1782015 INFO  (Block report processor) [    ] BlockStateChange BLOCK* processReport 0x4db8a2b9442b0: from storage DS-7d32bb5d-d5bf-4bba-b2e1-1e832aaed3aa node DatanodeRegistration(127.0.0.1:35831, datanodeUuid=34680c1c-4565-41f9-b121-30b0b593a0e2, infoPort=39177, infoSecurePort=0, ipcPort=35053, storageInfo=lv=-56;cid=testClusterID;nsid=1784438414;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 1782016 INFO  (Block report processor) [    ] BlockStateChange BLOCK* processReport 0x4db8a2b9442b0: from storage DS-6870fa44-2c89-47c1-be4a-00301f162a31 node DatanodeRegistration(127.0.0.1:35831, datanodeUuid=34680c1c-4565-41f9-b121-30b0b593a0e2, infoPort=39177, infoSecurePort=0, ipcPort=35053, storageInfo=lv=-56;cid=testClusterID;nsid=1784438414;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 1782074 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testNormalMove
   [junit4]   2> 1782075 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 1782075 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1782075 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1782075 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 1782075 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1782075 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 1782076 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T15:27:37-06:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1782077 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1782077 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1782077 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 1782077 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7712403d{/solr,null,AVAILABLE}
   [junit4]   2> 1782078 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1e835bcf{SSL,[ssl, http/1.1]}{127.0.0.1:38709}
   [junit4]   2> 1782078 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [    ] o.e.j.s.Server Started @1782123ms
   [junit4]   2> 1782078 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38709}
   [junit4]   2> 1782078 ERROR (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1782078 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1782078 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1782078 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1782078 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1782078 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-04-16T03:34:20.982762Z
   [junit4]   2> 1782079 INFO  (zkConnectionManagerCallback-1335-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1782080 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1782084 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34475/solr
   [junit4]   2> 1782085 INFO  (zkConnectionManagerCallback-1339-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1782086 DEBUG (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [n:127.0.0.1:38709_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$385/682972725@3b4e3cbb
   [junit4]   2> 1782087 INFO  (zkConnectionManagerCallback-1341-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1782089 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 1782089 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1782089 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [n:127.0.0.1:38709_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1782089 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 1782089 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1782089 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 1782090 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
   [junit4]   2> 1782090 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [n:127.0.0.1:38709_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1782090 DEBUG (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [n:127.0.0.1:38709_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/72147200464125973-127.0.0.1:38709_solr-n_0000000004
   [junit4]   2> 1782090 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/overseer_elect/election/72147200464125961-127.0.0.1:38863_solr-n_0000000003
   [junit4]   2> 1782091 DEBUG (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [n:127.0.0.1:38709_solr    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/72147200464125961-127.0.0.1:38863_solr-n_0000000003 to know if I could be the leader
   [junit4]   2> 1782091 DEBUG (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [n:127.0.0.1:38709_solr    ] o.a.s.c.ZkController Publish node=127.0.0.1:38709_solr as DOWN
   [junit4]   2> 1782091 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
   [junit4]   2> 1782091 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [n:127.0.0.1:38709_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1782091 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [n:127.0.0.1:38709_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38709_solr
   [junit4]   2> 1782091 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 1782091 DEBUG (OverseerStateUpdate-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:38709_solr"} current state version: 0
   [junit4]   2> 1782091 DEBUG (OverseerStateUpdate-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:38709_solr
   [junit4]   2> 1782091 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1782091 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1782091 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1782091 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1782091 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1782091 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1782091 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1782091 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1782091 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1782092 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1782092 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1782092 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1782092 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1782092 INFO  (zkCallback-1297-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1782092 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1782092 INFO  (zkCallback-1308-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1782092 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1782092 INFO  (zkCallback-1330-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1782092 INFO  (zkCallback-1295-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1782092 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1782092 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1782092 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1782092 INFO  (zkCallback-1330-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1782092 INFO  (zkCallback-1285-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1782092 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1782093 INFO  (zkCallback-1318-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1782093 INFO  (zkCallback-1313-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1782093 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1782093 INFO  (zkCallback-1340-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1782093 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1782093 INFO  (zkCallback-1283-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1782093 INFO  (zkCallback-1323-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1782192 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
   [junit4]   2> 1782192 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 1782273 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [n:127.0.0.1:38709_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38709.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3917c857
   [junit4]   2> 1782278 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [n:127.0.0.1:38709_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38709.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3917c857
   [junit4]   2> 1782278 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [n:127.0.0.1:38709_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38709.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3917c857
   [junit4]   2> 1782279 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [n:127.0.0.1:38709_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_1C49CA1791E3FFDD-001/tempDir-001/node5/.
   [junit4]   2> 1782281 INFO  (zkConnectionManagerCallback-1348-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1782282 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 1782282 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1782282 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [n:127.0.0.1:38709_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
   [junit4]   2> 1782283 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 1782283 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1782283 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 1782283 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [n:127.0.0.1:38709_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34475/solr ready
   [junit4]   2> 1782314 INFO  (zkConnectionManagerCallback-1350-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1782316 INFO  (TEST-MoveReplicaHDFSTest.testNormalMove-seed#[1C49CA1791E3FFDD]) [    ] o.a.s.c.MoveReplicaTest total_jettys: 5
   [junit4]   2> 1782324 INFO  (qtp1702062521-11170) [n:127.0.0.1:34547_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf1&maxShardsPerNode=2&autoAddReplicas=false&name=MoveReplicaHDFSTest_coll_false&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1782324 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/collection-queue-work
   [junit4]   2> 1782324 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/overseer/collection-queue-work/qnr-0000000002
   [junit4]   2> 1782325 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
   [junit4]   2> 1782325 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.ZkDistributedQueue Found child node with improper name: qnr-0000000002
   [junit4]   2> 1782325 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/collection-queue-work
   [junit4]   2> 1782325 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
   [junit4]   2> 1782325 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.ZkDistributedQueue Found child node with improper name: qnr-0000000002
   [junit4]   2> 1782325 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Returning topN elements: [/overseer/collection-queue-work/qn-0000000002, ]
   [junit4]   2> 1782325 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskProcessor Got 1 tasks from work-queue : [[org.apache.solr.cloud.OverseerTaskQueue$QueueEvent@2e684ed1]]
   [junit4]   2> 1782325 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskProcessor Marked task [/overseer/collection-queue-work/qn-0000000002] as running
   [junit4]   2> 1782325 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskProcessor Overseer Collection Message Handler: Get the message id:/overseer/collection-queue-work/qn-0000000002 message:{
   [junit4]   2>   "name":"MoveReplicaHDFSTest_coll_false",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"2",
   [junit4]   2>   "collection.configName":"conf1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "maxShardsPerNode":"2",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"2",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"}
   [junit4]   2> 1782326 DEBUG (OverseerThreadFactory-2106-thread-2) [    ] o.a.s.c.OverseerTaskProcessor Runner processing /overseer/collection-queue-work/qn-0000000002
   [junit4]   2> 1782326 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 1
   [junit4]   2> 1782326 DEBUG (OverseerThreadFactory-2106-thread-2) [    ] o.a.s.c.a.c.OverseerCollectionMessageHandler OverseerCollectionMessageHandler.processMessage : create , {
   [junit4]   2>   "name":"MoveReplicaHDFSTest_coll_false",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"2",
   [junit4]   2>   "collection.configName":"conf1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "maxShardsPerNode":"2",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"2",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"}
   [junit4]   2> 1782326 INFO  (OverseerThreadFactory-2106-thread-2) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection MoveReplicaHDFSTest_coll_false
   [junit4]   2> 1782326 INFO  (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1782326 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskProcessor RunningTasks: [/overseer/collection-queue-work/qn-0000000002]
   [junit4]   2> 1782326 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
   [junit4]   2> 1782326 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
   [junit4]   2> 1782326 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: [/overseer/collection-queue-work/qn-0000000002]
   [junit4]   2> 1782326 DEBUG (OverseerCollectionConfigSetProcessor-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Peeking for top 99 elements. ExcludeSet: [/overseer/collection-queue-work/qn-0000000002],[]
   [junit4]   2> 1782326 DEBUG (OverseerThreadFactory-2106-thread-2) [    ] o.a.s.c.a.c.Assign Identify nodes using default
   [junit4]   2> 1782326 DEBUG (OverseerThreadFactory-2106-thread-2) [    ] o.a.s.c.a.c.OverseerCollectionMessageHandler creating collections conf node /collections/MoveReplicaHDFSTest_coll_false 
   [junit4]   2> 1782327 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/collections
   [junit4]   2> 1782327 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/collections
   [junit4]   2> 1782327 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/collections
   [junit4]   2> 1782327 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/collections
   [junit4]   2> 1782327 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/collections
   [junit4]   2> 1782327 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/collections
   [junit4]   2> 1782327 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/collections
   [junit4]   2> 1782327 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/collections
   [junit4]   2> 1782327 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/collections
   [junit4]   2> 1782327 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/collections
   [junit4]   2> 1782327 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/collections
   [junit4]   2> 1782327 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1782327 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1782327 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1782327 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1782327 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1782327 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1782327 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1782327 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1782327 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1782327 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1782328 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1782328 DEBUG (OverseerThreadFactory-2106-thread-2) [    ] o.a.s.c.a.c.CreateCollectionCmd Check for collection zkNode:MoveReplicaHDFSTest_coll_false
   [junit4]   2> 1782328 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1782329 DEBUG (OverseerThreadFactory-2106-thread-2) [    ] o.a.s.c.a.c.CreateCollectionCmd Collection zkNode exists
   [junit4]   2> 1782329 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
   [junit4]   2> 1782330 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 1782330 DEBUG (OverseerStateUpdate-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "name":"MoveReplicaHDFSTest_coll_false",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"2",
   [junit4]   2>   "collection.configName":"conf1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "maxShardsPerNode":"2",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"2",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"} current state version: 0
   [junit4]   2> 1782330 DEBUG (OverseerStateUpdate-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.o.ClusterStateMutator building a new cName: MoveReplicaHDFSTest_coll_false
   [junit4]   2> 1782330 DEBUG (OverseerStateUpdate-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/MoveReplicaHDFSTest_coll_false/state.json
   [junit4]   2> 1782330 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
   [junit4]   2> 1782330 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 1782372 DEBUG (ScheduledTrigger-2104-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
   [junit4]   2> 1782430 DEBUG (OverseerThreadFactory-2106-thread-2) [    ] o.a.s.c.a.c.CreateCollectionCmd Creating SolrCores for new collection MoveReplicaHDFSTest_coll_false, shardNames [shard1, shard2] , message : {
   [junit4]   2>   "name":"MoveReplicaHDFSTest_coll_false",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"2",
   [junit4]   2>   "collection.configName":"conf1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "maxShardsPerNode":"2",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"2",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"}
   [junit4]   2> 1782431 DEBUG (OverseerThreadFactory-2106-thread-2) [    ] o.a.s.c.a.c.CreateCollectionCmd Creating core MoveReplicaHDFSTest_coll_false_shard1_replica_n1 as part of shard shard1 of collection MoveReplicaHDFSTest_coll_false on 127.0.0.1:38709_solr
   [junit4]   2> 1782431 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
   [junit4]   2> 1782432 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 1782432 DEBUG (OverseerStateUpdate-72147200464125963-127.0.0.1:34547_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"MoveReplicaHDFSTest_coll_false",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"MoveReplicaHDFSTest_coll_false_shard1_replica_n1",
   [junit4]   2>   "state":"down",
 

[...truncated too long message...]

erErrorCode = Session expired for /live_nodes]
   [junit4]   2> 1851179 INFO  (jetty-closer-1262-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5b6d46af{/solr,null,UNAVAILABLE}
   [junit4]   2> 1851179 INFO  (jetty-closer-1262-thread-2) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 1851179 ERROR (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1851180 INFO  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34475 34475
   [junit4]   2> 1851669 INFO  (Thread-1139) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34475 34475
   [junit4]   2> 1851669 WARN  (Thread-1139) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	34	/solr/collections/MoveReplicaHDFSTest_coll_false/terms/shard1
   [junit4]   2> 	28	/solr/collections/MoveReplicaHDFSTest_coll_true/terms/shard2
   [junit4]   2> 	18	/solr/configs/conf1
   [junit4]   2> 	17	/solr/aliases.json
   [junit4]   2> 	10	/solr/collections/MoveReplicaHDFSTest_coll_false/terms/shard2
   [junit4]   2> 	10	/solr/collections/MoveReplicaHDFSTest_failed_coll_false/terms/shard1
   [junit4]   2> 	10	/solr/collections/MoveReplicaHDFSTest_coll_true/terms/shard1
   [junit4]   2> 	9	/solr/collections/MoveReplicaHDFSTest_failed_coll_true/terms/shard1
   [junit4]   2> 	8	/solr/collections/MoveReplicaHDFSTest_failed_coll_false/terms/shard2
   [junit4]   2> 	7	/solr/collections/MoveReplicaHDFSTest_failed_coll_true/terms/shard2
   [junit4]   2> 	6	/solr/security.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	62	/solr/collections/MoveReplicaHDFSTest_coll_false/state.json
   [junit4]   2> 	58	/solr/collections/MoveReplicaHDFSTest_coll_true/state.json
   [junit4]   2> 	37	/solr/collections/MoveReplicaHDFSTest_failed_coll_true/state.json
   [junit4]   2> 	31	/solr/collections/MoveReplicaHDFSTest_failed_coll_false/state.json
   [junit4]   2> 	17	/solr/clusterprops.json
   [junit4]   2> 	17	/solr/clusterstate.json
   [junit4]   2> 	3	/solr/collections/MoveReplicaHDFSTest_coll_false/leader_elect/shard1/election/72147200464125961-core_node5-n_0000000000
   [junit4]   2> 	3	/solr/collections/MoveReplicaHDFSTest_coll_true/leader_elect/shard2/election/72147200464125961-core_node8-n_0000000000
   [junit4]   2> 	2	/solr/autoscaling.json
   [junit4]   2> 	2	/solr/overseer_elect/election/72147200464125963-127.0.0.1:34547_solr-n_0000000000
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	32	/solr/collections
   [junit4]   2> 	29	/solr/live_nodes
   [junit4]   2> 	2	/solr/overseer/queue
   [junit4]   2> 	2	/solr/autoscaling/events/.scheduled_maintenance
   [junit4]   2> 	2	/solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 	2	/solr/overseer/queue-work
   [junit4]   2> 	2	/solr/overseer/collection-queue-work
   [junit4]   2> 
   [junit4]   2> 1851670 WARN  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
   [junit4]   2> 1851708 INFO  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
   [junit4]   2> 1851809 WARN  (DataNode: [[[DISK]file:/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_1C49CA1791E3FFDD-001/tempDir-002/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_1C49CA1791E3FFDD-001/tempDir-002/hdfsBaseDir/data/data4/]]  heartbeating to localhost.localdomain/127.0.0.1:35827) [    ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
   [junit4]   2> 1851809 WARN  (DataNode: [[[DISK]file:/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_1C49CA1791E3FFDD-001/tempDir-002/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_1C49CA1791E3FFDD-001/tempDir-002/hdfsBaseDir/data/data4/]]  heartbeating to localhost.localdomain/127.0.0.1:35827) [    ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-791040628-88.99.242.108-1523849660458 (Datanode Uuid 34680c1c-4565-41f9-b121-30b0b593a0e2) service to localhost.localdomain/127.0.0.1:35827
   [junit4]   2> 1851810 WARN  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
   [junit4]   2> 1851814 INFO  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
   [junit4]   2> 1851915 WARN  (DataNode: [[[DISK]file:/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_1C49CA1791E3FFDD-001/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_1C49CA1791E3FFDD-001/tempDir-002/hdfsBaseDir/data/data2/]]  heartbeating to localhost.localdomain/127.0.0.1:35827) [    ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
   [junit4]   2> 1851915 WARN  (DataNode: [[[DISK]file:/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_1C49CA1791E3FFDD-001/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_1C49CA1791E3FFDD-001/tempDir-002/hdfsBaseDir/data/data2/]]  heartbeating to localhost.localdomain/127.0.0.1:35827) [    ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-791040628-88.99.242.108-1523849660458 (Datanode Uuid 28cdadbc-ad1f-4d89-9e30-a8dc5075f1a0) service to localhost.localdomain/127.0.0.1:35827
   [junit4]   2> 1851919 INFO  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:0
   [junit4]   2> 1852027 INFO  (SUITE-MoveReplicaHDFSTest-seed#[1C49CA1791E3FFDD]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34475 34475
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_1C49CA1791E3FFDD-001
   [junit4]   2> Apr 16, 2018 3:35:30 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 66 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {id=PostingsFormat(name=MockRandom)}, docValues:{}, maxPointsInLeafNode=168, maxMBSortInHeap=6.242076115612907, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@7a79bcb5), locale=sq-XK, timezone=America/Managua
   [junit4]   2> NOTE: Linux 4.13.0-37-generic amd64/Oracle Corporation 10 (64-bit)/cpus=8,threads=2,free=300341624,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [TestPayloadScoreQParserPlugin, TestLMDirichletSimilarityFactory, TestCrossCoreJoin, SynonymTokenizerTest, NodeMarkersRegistrationTest, SolrRequestParserTest, CreateRoutedAliasTest, IndexBasedSpellCheckerTest, HdfsChaosMonkeyNothingIsSafeTest, DefaultValueUpdateProcessorTest, SolrCloudExampleTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, TestHighFrequencyDictionaryFactory, TestSimDistributedQueue, TestSubQueryTransformerDistrib, ShardRoutingTest, SolrGraphiteReporterTest, TestReplicationHandlerBackup, TestHashPartitioner, FullSolrCloudDistribCmdsTest, HdfsAutoAddReplicasIntegrationTest, BasicDistributedZkTest, VMParamsZkACLAndCredentialsProvidersTest, TestPHPSerializedResponseWriter, TestOmitPositions, TestExceedMaxTermLength, TestNoOpRegenerator, RequestHandlersTest, DeleteNodeTest, XmlUpdateRequestHandlerTest, TestApiFramework, TestCloudJSONFacetJoinDomain, SolrCoreCheckLockOnStartupTest, BadCopyFieldTest, SpellCheckCollatorTest, ConfigSetsAPITest, TestCollationFieldDocValues, DistributedFacetPivotLongTailTest, MoveReplicaHDFSTest]
   [junit4] Completed [456/802 (1!)] on J0 in 79.92s, 4 tests, 1 error <<< FAILURES!

[...truncated 44931 lines...]
[repro] Jenkins log URL: https://jenkins.thetaphi.de/job/Lucene-Solr-BadApples-master-Linux/23/consoleText

[repro] Revision: dad2d1076db632535c33fa118eb851ad7d0e2537

[repro] Repro line:  ant test  -Dtestcase=MoveReplicaHDFSTest -Dtests.method=testFailedMove -Dtests.seed=1C49CA1791E3FFDD -Dtests.multiplier=3 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=sq-XK -Dtests.timezone=America/Managua -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]    solr/core
[repro]       MoveReplicaHDFSTest
[repro] ant compile-test

[...truncated 3303 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.MoveReplicaHDFSTest" -Dtests.showOutput=onerror  -Dtests.seed=1C49CA1791E3FFDD -Dtests.multiplier=3 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=sq-XK -Dtests.timezone=America/Managua -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[...truncated 91 lines...]
[repro] Failures:
[repro]   0/5 failed: org.apache.solr.cloud.MoveReplicaHDFSTest
[repro] Exiting with code 0

[...truncated 40 lines...]

[JENKINS] Lucene-Solr-BadApples-master-Linux (32bit/jdk1.8.0_162) - Build # 22 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-BadApples-master-Linux/22/
Java: 32bit/jdk1.8.0_162 -server -XX:+UseParallelGC

7 tests failed.
FAILED:  org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove

Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:45405/solr/MoveReplicaHDFSTest_failed_coll_true, http://127.0.0.1:38317/solr/MoveReplicaHDFSTest_failed_coll_true]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:45405/solr/MoveReplicaHDFSTest_failed_coll_true, http://127.0.0.1:38317/solr/MoveReplicaHDFSTest_failed_coll_true]
	at __randomizedtesting.SeedInfo.seed([985392D38082195A:329E41213751CC8A]:0)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:462)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:993)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
	at org.apache.solr.cloud.MoveReplicaTest.testFailedMove(MoveReplicaTest.java:310)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:38317/solr/MoveReplicaHDFSTest_failed_coll_true: no servers hosting shard: shard1
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:436)
	... 46 more


FAILED:  org.apache.solr.search.TestSolr4Spatial2.testLatLonRetrieval

Error Message:
Path not found: response/docs/[14]/llp_N_dv_dvasst

Stack Trace:
java.lang.RuntimeException: Path not found: response/docs/[14]/llp_N_dv_dvasst
	at __randomizedtesting.SeedInfo.seed([985392D38082195A:BEA7F82F8C255F14]:0)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:1004)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:951)
	at org.apache.solr.search.TestSolr4Spatial2.testLatLonRetrieval(TestSolr4Spatial2.java:226)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.search.TestSolr4Spatial2.testLatLonRetrieval

Error Message:
mismatch: '-40,40'!='-40,39.99999997' @ response/docs/[5]/llp_N_dv_dvasst/[0]

Stack Trace:
java.lang.RuntimeException: mismatch: '-40,40'!='-40,39.99999997' @ response/docs/[5]/llp_N_dv_dvasst/[0]
	at __randomizedtesting.SeedInfo.seed([985392D38082195A:BEA7F82F8C255F14]:0)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:1004)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:951)
	at org.apache.solr.search.TestSolr4Spatial2.testLatLonRetrieval(TestSolr4Spatial2.java:226)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.search.TestSolr4Spatial2.testLatLonRetrieval

Error Message:
Path not found: response/docs/[14]/llp_N_dv_dvasst

Stack Trace:
java.lang.RuntimeException: Path not found: response/docs/[14]/llp_N_dv_dvasst
	at __randomizedtesting.SeedInfo.seed([985392D38082195A:BEA7F82F8C255F14]:0)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:1004)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:951)
	at org.apache.solr.search.TestSolr4Spatial2.testLatLonRetrieval(TestSolr4Spatial2.java:226)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.search.TestSolr4Spatial2.testLatLonRetrieval

Error Message:
Path not found: response/docs/[23]/llp_N_dv_dvasst

Stack Trace:
java.lang.RuntimeException: Path not found: response/docs/[23]/llp_N_dv_dvasst
	at __randomizedtesting.SeedInfo.seed([985392D38082195A:BEA7F82F8C255F14]:0)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:1004)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:951)
	at org.apache.solr.search.TestSolr4Spatial2.testLatLonRetrieval(TestSolr4Spatial2.java:226)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.search.TestSolr4Spatial2.testLatLonRetrieval

Error Message:
mismatch: '-40,40'!='-40,39.99999997' @ response/docs/[5]/llp_N_dv_dvasst/[0]

Stack Trace:
java.lang.RuntimeException: mismatch: '-40,40'!='-40,39.99999997' @ response/docs/[5]/llp_N_dv_dvasst/[0]
	at __randomizedtesting.SeedInfo.seed([985392D38082195A:BEA7F82F8C255F14]:0)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:1004)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:951)
	at org.apache.solr.search.TestSolr4Spatial2.testLatLonRetrieval(TestSolr4Spatial2.java:226)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.search.TestSolr4Spatial2.testLatLonRetrieval

Error Message:
mismatch: '-40,40'!='-40,39.99999997' @ response/docs/[5]/llp_N_dv_dvasst/[0]

Stack Trace:
java.lang.RuntimeException: mismatch: '-40,40'!='-40,39.99999997' @ response/docs/[5]/llp_N_dv_dvasst/[0]
	at __randomizedtesting.SeedInfo.seed([985392D38082195A:BEA7F82F8C255F14]:0)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:1004)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:951)
	at org.apache.solr.search.TestSolr4Spatial2.testLatLonRetrieval(TestSolr4Spatial2.java:226)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12378 lines...]
   [junit4] Suite: org.apache.solr.search.TestSolr4Spatial2
   [junit4]   2> 59611 INFO  (SUITE-TestSolr4Spatial2-seed#[985392D38082195A]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J1/temp/solr.search.TestSolr4Spatial2_985392D38082195A-001/init-core-data-001
   [junit4]   2> 59612 WARN  (SUITE-TestSolr4Spatial2-seed#[985392D38082195A]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
   [junit4]   2> 59612 INFO  (SUITE-TestSolr4Spatial2-seed#[985392D38082195A]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 59613 INFO  (SUITE-TestSolr4Spatial2-seed#[985392D38082195A]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 59621 INFO  (SUITE-TestSolr4Spatial2-seed#[985392D38082195A]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 59621 INFO  (SUITE-TestSolr4Spatial2-seed#[985392D38082195A]-worker) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 59642 INFO  (SUITE-TestSolr4Spatial2-seed#[985392D38082195A]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 59665 INFO  (SUITE-TestSolr4Spatial2-seed#[985392D38082195A]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=test
   [junit4]   2> 59673 WARN  (SUITE-TestSolr4Spatial2-seed#[985392D38082195A]-worker) [    ] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.TrieFloatField]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 59677 WARN  (SUITE-TestSolr4Spatial2-seed#[985392D38082195A]-worker) [    ] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.TrieDoubleField]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 59685 WARN  (SUITE-TestSolr4Spatial2-seed#[985392D38082195A]-worker) [    ] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.SpatialTermQueryPrefixTreeFieldType]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 59691 WARN  (SUITE-TestSolr4Spatial2-seed#[985392D38082195A]-worker) [    ] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.SpatialPointVectorFieldType]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 59729 INFO  (SUITE-TestSolr4Spatial2-seed#[985392D38082195A]-worker) [    ] o.a.s.s.IndexSchema Loaded schema test/1.4 with uniqueid field id
   [junit4]   2> 59773 INFO  (SUITE-TestSolr4Spatial2-seed#[985392D38082195A]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@d726a1
   [junit4]   2> 59782 INFO  (SUITE-TestSolr4Spatial2-seed#[985392D38082195A]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@d726a1
   [junit4]   2> 59782 INFO  (SUITE-TestSolr4Spatial2-seed#[985392D38082195A]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@d726a1
   [junit4]   2> 59783 INFO  (coreLoadExecutor-100-thread-1) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 59784 INFO  (coreLoadExecutor-100-thread-1) [    x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 59814 INFO  (coreLoadExecutor-100-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 59820 INFO  (coreLoadExecutor-100-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 59829 INFO  (coreLoadExecutor-100-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.4 with uniqueid field id
   [junit4]   2> 59829 INFO  (coreLoadExecutor-100-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/core/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 59829 INFO  (coreLoadExecutor-100-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@d726a1
   [junit4]   2> 59830 INFO  (coreLoadExecutor-100-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 59830 INFO  (coreLoadExecutor-100-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/core/src/test-files/solr/collection1], dataDir=[/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J1/temp/solr.search.TestSolr4Spatial2_985392D38082195A-001/init-core-data-001/]
   [junit4]   2> 59831 INFO  (coreLoadExecutor-100-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=21, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 59872 INFO  (coreLoadExecutor-100-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 59872 INFO  (coreLoadExecutor-100-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 59873 INFO  (coreLoadExecutor-100-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=21, maxMergeAtOnceExplicit=22, maxMergedSegmentMB=31.3701171875, floorSegmentMB=0.49609375, forceMergeDeletesPctAllowed=27.815214827872186, segmentsPerTier=48.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 59874 INFO  (coreLoadExecutor-100-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@d79554[collection1] main]
   [junit4]   2> 59874 WARN  (coreLoadExecutor-100-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/core/src/test-files/solr/collection1/conf; switching to use InMemory storage instead.
   [junit4]   2> 59875 INFO  (coreLoadExecutor-100-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 59875 INFO  (searcherExecutor-101-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@d79554[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 59876 INFO  (SUITE-TestSolr4Spatial2-seed#[985392D38082195A]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 59878 INFO  (TEST-TestSolr4Spatial2.testLatLonRetrieval-seed#[985392D38082195A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testLatLonRetrieval
   [junit4]   2> 59878 INFO  (TEST-TestSolr4Spatial2.testLatLonRetrieval-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 59879 INFO  (TEST-TestSolr4Spatial2.testLatLonRetrieval-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:*} 0 0
   [junit4]   2> 59894 INFO  (TEST-TestSolr4Spatial2.testLatLonRetrieval-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[5]} 0 13
   [junit4]   2> 59897 INFO  (TEST-TestSolr4Spatial2.testLatLonRetrieval-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[7]} 0 0
   [junit4]   2> 59897 INFO  (TEST-TestSolr4Spatial2.testLatLonRetrieval-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[8]} 0 0
   [junit4]   2> 59898 INFO  (TEST-TestSolr4Spatial2.testLatLonRetrieval-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 59898 INFO  (TEST-TestSolr4Spatial2.testLatLonRetrieval-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 59899 INFO  (TEST-TestSolr4Spatial2.testLatLonRetrieval-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[0]} 0 0
   [junit4]   2> 59899 INFO  (TEST-TestSolr4Spatial2.testLatLonRetrieval-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[4]} 0 0
   [junit4]   2> 59899 INFO  (TEST-TestSolr4Spatial2.testLatLonRetrieval-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 59899 INFO  (TEST-TestSolr4Spatial2.testLatLonRetrieval-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[6]} 0 0
   [junit4]   2> 59900 INFO  (TEST-TestSolr4Spatial2.testLatLonRetrieval-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 59900 INFO  (TEST-TestSolr4Spatial2.testLatLonRetrieval-seed#[985392D38082195A]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1d596b0 commitCommandVersion:0
   [junit4]   2> 59909 INFO  (TEST-TestSolr4Spatial2.testLatLonRetrieval-seed#[985392D38082195A]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@509c08[collection1] main]
   [junit4]   2> 59909 INFO  (TEST-TestSolr4Spatial2.testLatLonRetrieval-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 59910 INFO  (searcherExecutor-101-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@509c08[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):c9)))}
   [junit4]   2> 59910 INFO  (TEST-TestSolr4Spatial2.testLatLonRetrieval-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 10
   [junit4]   2> 59921 INFO  (TEST-TestSolr4Spatial2.testLatLonRetrieval-seed#[985392D38082195A]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&fl=*&sort=id+asc&wt=xml} hits=9 status=0 QTime=9
   [junit4]   2> 59924 ERROR (TEST-TestSolr4Spatial2.testLatLonRetrieval-seed#[985392D38082195A]) [    ] o.a.s.SolrTestCaseJ4 query failed JSON validation. error=mismatch: '-40,40'!='-40,39.99999997' @ response/docs/[5]/llp_N_dv_dvasst/[0]
   [junit4]   2>  expected =response/docs/[5]/llp_N_dv_dvasst==['-40,40']
   [junit4]   2>  response = {
   [junit4]   2>   "responseHeader":{
   [junit4]   2>     "status":0,
   [junit4]   2>     "QTime":9},
   [junit4]   2>   "response":{"numFound":9,"start":0,"docs":[
   [junit4]   2>       {
   [junit4]   2>         "id":"0",
   [junit4]   2>         "llp_1_dv_st":"40.2996543270,-74.0824956673"},
   [junit4]   2>       {
   [junit4]   2>         "id":"1",
   [junit4]   2>         "llp_N_dv_st":["-40,40",
   [junit4]   2>           "-45,45"]},
   [junit4]   2>       {
   [junit4]   2>         "id":"2",
   [junit4]   2>         "llp_N_dv_st":["-40,40"]},
   [junit4]   2>       {
   [junit4]   2>         "id":"3",
   [junit4]   2>         "llp_1_dv_dvasst":"40.29965431,-74.08249572"},
   [junit4]   2>       {
   [junit4]   2>         "id":"4",
   [junit4]   2>         "llp_N_dv_dvasst":["-45,45",
   [junit4]   2>           "-40,39.99999997"]},
   [junit4]   2>       {
   [junit4]   2>         "id":"5",
   [junit4]   2>         "llp_N_dv_dvasst":["-40,39.99999997"]},
   [junit4]   2>       {
   [junit4]   2>         "id":"6",
   [junit4]   2>         "llp_N_dv_dvasst":["-90,179.99999992",
   [junit4]   2>           "-90,-180",
   [junit4]   2>           "0,0",
   [junit4]   2>           "0,179.99999992",
   [junit4]   2>           "0,-180",
   [junit4]   2>           "89.99999996,0",
   [junit4]   2>           "89.99999996,179.99999992",
   [junit4]   2>           "89.99999996,-180"]},
   [junit4]   2>       {
   [junit4]   2>         "id":"7"},
   [junit4]   2>       {
   [junit4]   2>         "id":"8"}]
   [junit4]   2>   }}
   [junit4]   2> 
   [junit4]   2>  request = q=*:*&fl=*&sort=id+asc&wt=xml
   [junit4]   2> 59925 INFO  (TEST-TestSolr4Spatial2.testLatLonRetrieval-seed#[985392D38082195A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testLatLonRetrieval
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSolr4Spatial2 -Dtests.method=testLatLonRetrieval -Dtests.seed=985392D38082195A -Dtests.multiplier=3 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=sr-Latn -Dtests.timezone=Etc/GMT0 -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.06s J1 | TestSolr4Spatial2.testLatLonRetrieval <<<
   [junit4]    > Throwable #1: java.lang.RuntimeException: mismatch: '-40,40'!='-40,39.99999997' @ response/docs/[5]/llp_N_dv_dvasst/[0]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([985392D38082195A:BEA7F82F8C255F14]:0)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:1004)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:951)
   [junit4]    > 	at org.apache.solr.search.TestSolr4Spatial2.testLatLonRetrieval(TestSolr4Spatial2.java:226)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 59944 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testLLPDecodeIsStableAndPrecise {seed=[985392D38082195A:DBAF489A98DFF7E2]}
   [junit4]   2> 59944 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 59944 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:*} 0 0
   [junit4]   2> 59952 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[0]} 0 0
   [junit4]   2> 59952 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 59952 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1d596b0 commitCommandVersion:0
   [junit4]   2> 59956 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@b2be8[collection1] main]
   [junit4]   2> 59957 INFO  (searcherExecutor-101-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@b2be8[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_1(8.0.0):c1)))}
   [junit4]   2> 59958 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 59958 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 5
   [junit4]   2> 59958 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=/select params={q=id:0} hits=1 status=0 QTime=0
   [junit4]   2> 59959 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[0]} 0 0
   [junit4]   2> 59959 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 59959 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1d596b0 commitCommandVersion:0
   [junit4]   2> 59961 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@550dd3[collection1] main]
   [junit4]   2> 59963 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 59963 INFO  (searcherExecutor-101-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@550dd3[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_2(8.0.0):c1)))}
   [junit4]   2> 59963 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 3
   [junit4]   2> 59964 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=/select params={q=id:0} hits=1 status=0 QTime=0
   [junit4]   2> 59964 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testLLPDecodeIsStableAndPrecise {seed=[985392D38082195A:DBAF489A98DFF7E2]}
   [junit4]   2> 59967 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testLLPDecodeIsStableAndPrecise {seed=[985392D38082195A:6FF9F466AC1D3CCE]}
   [junit4]   2> 59967 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 59967 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:*} 0 0
   [junit4]   2> 59968 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[0]} 0 0
   [junit4]   2> 59968 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 59968 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1d596b0 commitCommandVersion:0
   [junit4]   2> 59970 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@1b17d4a[collection1] main]
   [junit4]   2> 59971 INFO  (searcherExecutor-101-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1b17d4a[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_3(8.0.0):c1)))}
   [junit4]   2> 59971 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 59972 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 3
   [junit4]   2> 59972 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=/select params={q=id:0} hits=1 status=0 QTime=0
   [junit4]   2> 59973 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[0]} 0 0
   [junit4]   2> 59973 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 59973 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1d596b0 commitCommandVersion:0
   [junit4]   2> 59975 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@1cf4e69[collection1] main]
   [junit4]   2> 59978 INFO  (searcherExecutor-101-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1cf4e69[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_4(8.0.0):c1)))}
   [junit4]   2> 59978 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 59978 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 4
   [junit4]   2> 59978 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=/select params={q=id:0} hits=1 status=0 QTime=0
   [junit4]   2> 59979 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testLLPDecodeIsStableAndPrecise {seed=[985392D38082195A:6FF9F466AC1D3CCE]}
   [junit4]   2> 59981 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testLLPDecodeIsStableAndPrecise {seed=[985392D38082195A:E11062BAFDD97405]}
   [junit4]   2> 59981 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 59981 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:*} 0 0
   [junit4]   2> 59982 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[0]} 0 0
   [junit4]   2> 59982 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 59982 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1d596b0 commitCommandVersion:0
   [junit4]   2> 59984 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@15c4835[collection1] main]
   [junit4]   2> 59990 INFO  (searcherExecutor-101-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@15c4835[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_5(8.0.0):c1)))}
   [junit4]   2> 59990 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 59990 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 7
   [junit4]   2> 59990 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=/select params={q=id:0} hits=1 status=0 QTime=0
   [junit4]   2> 59994 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[0]} 0 3
   [junit4]   2> 59994 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 59994 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1d596b0 commitCommandVersion:0
   [junit4]   2> 59997 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@511[collection1] main]
   [junit4]   2> 59998 INFO  (searcherExecutor-101-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@511[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_6(8.0.0):c1)))}
   [junit4]   2> 59999 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 59999 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 4
   [junit4]   2> 59999 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=/select params={q=id:0} hits=1 status=0 QTime=0
   [junit4]   2> 60000 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testLLPDecodeIsStableAndPrecise {seed=[985392D38082195A:E11062BAFDD97405]}
   [junit4]   2> 60002 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testLLPDecodeIsStableAndPrecise {seed=[985392D38082195A:D0FEC95F91272F2C]}
   [junit4]   2> 60002 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 60002 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:*} 0 0
   [junit4]   2> 60003 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[0]} 0 0
   [junit4]   2> 60003 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 60003 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1d596b0 commitCommandVersion:0
   [junit4]   2> 60006 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@5aff7e[collection1] main]
   [junit4]   2> 60006 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 60007 INFO  (searcherExecutor-101-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5aff7e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_7(8.0.0):c1)))}
   [junit4]   2> 60008 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 4
   [junit4]   2> 60008 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=/select params={q=id:0} hits=1 status=0 QTime=0
   [junit4]   2> 60009 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[0]} 0 0
   [junit4]   2> 60009 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 60009 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1d596b0 commitCommandVersion:0
   [junit4]   2> 60012 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@1ca1ea5[collection1] main]
   [junit4]   2> 60012 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 60013 INFO  (searcherExecutor-101-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1ca1ea5[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_8(8.0.0):c1)))}
   [junit4]   2> 60014 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 4
   [junit4]   2> 60014 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=/select params={q=id:0} hits=1 status=0 QTime=0
   [junit4]   2> 60015 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testLLPDecodeIsStableAndPrecise {seed=[985392D38082195A:D0FEC95F91272F2C]}
   [junit4]   2> 60017 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testLLPDecodeIsStableAndPrecise {seed=[985392D38082195A:9C3F4CF2302FEF97]}
   [junit4]   2> 60017 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 60018 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:*} 0 0
   [junit4]   2> 60019 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[0]} 0 0
   [junit4]   2> 60019 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 60019 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1d596b0 commitCommandVersion:0
   [junit4]   2> 60023 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@61d5d3[collection1] main]
   [junit4]   2> 60023 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 60025 INFO  (searcherExecutor-101-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@61d5d3[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_9(8.0.0):c1)))}
   [junit4]   2> 60025 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 5
   [junit4]   2> 60025 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=/select params={q=id:0} hits=1 status=0 QTime=0
   [junit4]   2> 60026 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[0]} 0 0
   [junit4]   2> 60026 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 60026 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1d596b0 commitCommandVersion:0
   [junit4]   2> 60028 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@15877d1[collection1] main]
   [junit4]   2> 60028 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 60030 INFO  (searcherExecutor-101-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@15877d1[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_a(8.0.0):c1)))}
   [junit4]   2> 60030 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 3
   [junit4]   2> 60030 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=/select params={q=id:0} hits=1 status=0 QTime=0
   [junit4]   2> 60031 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testLLPDecodeIsStableAndPrecise {seed=[985392D38082195A:9C3F4CF2302FEF97]}
   [junit4]   2> 60033 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testLLPDecodeIsStableAndPrecise {seed=[985392D38082195A:DC59FAD4D93A297]}
   [junit4]   2> 60033 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 60033 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:*} 0 0
   [junit4]   2> 60034 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[0]} 0 0
   [junit4]   2> 60034 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 60034 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1d596b0 commitCommandVersion:0
   [junit4]   2> 60036 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@13348e1[collection1] main]
   [junit4]   2> 60037 INFO  (searcherExecutor-101-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@13348e1[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_b(8.0.0):c1)))}
   [junit4]   2> 60037 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 60037 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 3
   [junit4]   2> 60038 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=/select params={q=id:0} hits=1 status=0 QTime=0
   [junit4]   2> 60039 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[0]} 0 0
   [junit4]   2> 60039 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 60039 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1d596b0 commitCommandVersion:0
   [junit4]   2> 60041 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@9db8b[collection1] main]
   [junit4]   2> 60042 INFO  (searcherExecutor-101-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@9db8b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_c(8.0.0):c1)))}
   [junit4]   2> 60043 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 60043 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 3
   [junit4]   2> 60043 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=/select params={q=id:0} hits=1 status=0 QTime=0
   [junit4]   2> 60044 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testLLPDecodeIsStableAndPrecise {seed=[985392D38082195A:DC59FAD4D93A297]}
   [junit4]   2> 60049 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testLLPDecodeIsStableAndPrecise {seed=[985392D38082195A:331BFB2B5FA3B291]}
   [junit4]   2> 60050 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 60050 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:*} 0 0
   [junit4]   2> 60056 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[0]} 0 6
   [junit4]   2> 60057 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 60057 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1d596b0 commitCommandVersion:0
   [junit4]   2> 60058 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@1f6b411[collection1] main]
   [junit4]   2> 60060 INFO  (searcherExecutor-101-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1f6b411[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_d(8.0.0):c1)))}
   [junit4]   2> 60060 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 60060 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 3
   [junit4]   2> 60061 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=/select params={q=id:0} hits=1 status=0 QTime=0
   [junit4]   2> 60065 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[0]} 0 4
   [junit4]   2> 60066 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 60066 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1d596b0 commitCommandVersion:0
   [junit4]   2> 60068 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@16d3d39[collection1] main]
   [junit4]   2> 60069 INFO  (searcherExecutor-101-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@16d3d39[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_e(8.0.0):c1)))}
   [junit4]   2> 60069 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 60070 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 3
   [junit4]   2> 60070 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=/select params={q=id:0} hits=1 status=0 QTime=0
   [junit4]   2> 60071 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testLLPDecodeIsStableAndPrecise {seed=[985392D38082195A:331BFB2B5FA3B291]}
   [junit4]   2> 60073 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testLLPDecodeIsStableAndPrecise {seed=[985392D38082195A:AFA861517CB7263F]}
   [junit4]   2> 60073 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 60073 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:*} 0 0
   [junit4]   2> 60075 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[0]} 0 1
   [junit4]   2> 60075 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 60075 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1d596b0 commitCommandVersion:0
   [junit4]   2> 60077 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@19c3736[collection1] main]
   [junit4]   2> 60080 INFO  (searcherExecutor-101-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@19c3736[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_f(8.0.0):c1)))}
   [junit4]   2> 60080 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 60080 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 4
   [junit4]   2> 60081 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=/select params={q=id:0} hits=1 status=0 QTime=0
   [junit4]   2> 60082 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[0]} 0 0
   [junit4]   2> 60082 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 60082 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1d596b0 commitCommandVersion:0
   [junit4]   2> 60085 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@eff4a8[collection1] main]
   [junit4]   2> 60087 INFO  (searcherExecutor-101-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@eff4a8[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_g(8.0.0):c1)))}
   [junit4]   2> 60087 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 60087 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 4
   [junit4]   2> 60087 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=/select params={q=id:0} hits=1 status=0 QTime=0
   [junit4]   2> 60088 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testLLPDecodeIsStableAndPrecise {seed=[985392D38082195A:AFA861517CB7263F]}
   [junit4]   2> 60101 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testLLPDecodeIsStableAndPrecise {seed=[985392D38082195A:9D04843F0B7C3165]}
   [junit4]   2> 60101 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 60101 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:*} 0 0
   [junit4]   2> 60102 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[0]} 0 0
   [junit4]   2> 60103 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 60104 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1d596b0 commitCommandVersion:0
   [junit4]   2> 60108 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@1a6c4ca[collection1] main]
   [junit4]   2> 60109 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 60110 INFO  (searcherExecutor-101-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1a6c4ca[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_h(8.0.0):c1)))}
   [junit4]   2> 60114 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 10
   [junit4]   2> 60115 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=/select params={q=id:0} hits=1 status=0 QTime=0
   [junit4]   2> 60117 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[0]} 0 2
   [junit4]   2> 60131 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 60131 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1d596b0 commitCommandVersion:0
   [junit4]   2> 60134 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@436fde[collection1] main]
   [junit4]   2> 60134 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 60135 INFO  (searcherExecutor-101-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@436fde[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_i(8.0.0):c1)))}
   [junit4]   2> 60135 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 3
   [junit4]   2> 60135 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=/select params={q=id:0} hits=1 status=0 QTime=0
   [junit4]   2> 60136 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testLLPDecodeIsStableAndPrecise {seed=[985392D38082195A:9D04843F0B7C3165]}
   [junit4]   2> 60138 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testLLPDecodeIsStableAndPrecise {seed=[985392D38082195A:4A8FD2856F2B0637]}
   [junit4]   2> 60138 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 60138 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:*} 0 0
   [junit4]   2> 60139 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[0]} 0 0
   [junit4]   2> 60139 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 60139 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1d596b0 commitCommandVersion:0
   [junit4]   2> 60141 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@1ef5d4e[collection1] main]
   [junit4]   2> 60142 INFO  (searcherExecutor-101-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1ef5d4e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_j(8.0.0):c1)))}
   [junit4]   2> 60142 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 60142 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 3
   [junit4]   2> 60143 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=/select params={q=id:0} hits=1 status=0 QTime=0
   [junit4]   2> 60144 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[0]} 0 0
   [junit4]   2> 60144 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 60144 INFO  (TEST-TestSolr4Spatial2.testLLPDecodeIsStableAndPrecise-seed#[985392D38082195A]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1d596b0 commitCommandVersion:0
   [junit4]   2> 60146 INFO 

[...truncated too long message...]

]} 0 0
   [junit4]   2> 116134 INFO  (TEST-TestSolr4Spatial2.testBBox-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 116135 INFO  (TEST-TestSolr4Spatial2.testBBox-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 116135 INFO  (TEST-TestSolr4Spatial2.testBBox-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 116135 INFO  (TEST-TestSolr4Spatial2.testBBox-seed#[985392D38082195A]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@c1186f commitCommandVersion:0
   [junit4]   2> 116137 INFO  (TEST-TestSolr4Spatial2.testBBox-seed#[985392D38082195A]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@d33dc2[collection1] main]
   [junit4]   2> 116138 INFO  (TEST-TestSolr4Spatial2.testBBox-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 116139 INFO  (searcherExecutor-202-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@d33dc2[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_q(8.0.0):c3)))}
   [junit4]   2> 116139 INFO  (TEST-TestSolr4Spatial2.testBBox-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 3
   [junit4]   2> 116140 INFO  (TEST-TestSolr4Spatial2.testBBox-seed#[985392D38082195A]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q={!field+f%3DbboxD_dynamic+filter%3Dfalse+score%3DoverlapRatio+queryTargetProportion%3D0.25}Intersects(ENVELOPE(10,25,12,10))&debug=results&fl=*,score&wt=xml} hits=3 status=0 QTime=0
   [junit4]   2> 116141 INFO  (TEST-TestSolr4Spatial2.testBBox-seed#[985392D38082195A]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q={!field+f%3DbboxD_dynamic+filter%3Dfalse+score%3DoverlapRatio+queryTargetProportion%3D0.5+minSideLength%3D1}Intersects(ENVELOPE(0,0,12,12))&debug=results&fl=id,score&wt=xml} hits=3 status=0 QTime=0
   [junit4]   2> 116142 INFO  (TEST-TestSolr4Spatial2.testBBox-seed#[985392D38082195A]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q={!field+f%3DbboxD_dynamic+filter%3Dfalse+score%3Darea2D}Intersects(ENVELOPE(0,0,12,12))&debug=results&fl=id,score&wt=xml} hits=3 status=0 QTime=0
   [junit4]   2> 116142 INFO  (TEST-TestSolr4Spatial2.testBBox-seed#[985392D38082195A]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q={!field+f%3DbboxD_dynamic+filter%3Dfalse+score%3Darea}Intersects(ENVELOPE(0,0,12,12))&debug=results&fl=id,score&wt=xml} hits=3 status=0 QTime=0
   [junit4]   2> 116143 INFO  (TEST-TestSolr4Spatial2.testBBox-seed#[985392D38082195A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testBBox
   [junit4] OK      0.01s J1 | TestSolr4Spatial2.testBBox
   [junit4]   2> 116145 INFO  (TEST-TestSolr4Spatial2.testBadScoreParam-seed#[985392D38082195A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testBadScoreParam
   [junit4]   2> 116145 INFO  (TEST-TestSolr4Spatial2.testBadScoreParam-seed#[985392D38082195A]) [    ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 116145 INFO  (TEST-TestSolr4Spatial2.testBadScoreParam-seed#[985392D38082195A]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:*} 0 0
   [junit4]   2> 116145 INFO  (TEST-TestSolr4Spatial2.testBadScoreParam-seed#[985392D38082195A]) [    ] o.a.s.h.RequestHandlerBase Ignoring exception matching .
   [junit4]   2> 116145 INFO  (TEST-TestSolr4Spatial2.testBadScoreParam-seed#[985392D38082195A]) [    ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q={!field+f%3Dbbox+filter%3Dfalse+score%3Dbogus}Intersects(ENVELOPE(0,0,12,12))&qt=&start=0&rows=20&version=2.2} status=400 QTime=0
   [junit4]   2> 116146 INFO  (TEST-TestSolr4Spatial2.testBadScoreParam-seed#[985392D38082195A]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testBadScoreParam
   [junit4] OK      0.00s J1 | TestSolr4Spatial2.testBadScoreParam
   [junit4]   2> 116146 INFO  (SUITE-TestSolr4Spatial2-seed#[985392D38082195A]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 116146 INFO  (SUITE-TestSolr4Spatial2-seed#[985392D38082195A]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=15753098
   [junit4]   2> 116146 INFO  (SUITE-TestSolr4Spatial2-seed#[985392D38082195A]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 116146 INFO  (SUITE-TestSolr4Spatial2-seed#[985392D38082195A]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@570b10: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@1ed9af3
   [junit4]   2> 116149 INFO  (SUITE-TestSolr4Spatial2-seed#[985392D38082195A]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 116149 INFO  (SUITE-TestSolr4Spatial2-seed#[985392D38082195A]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@12b4dcb: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@19f056b
   [junit4]   2> 116151 INFO  (SUITE-TestSolr4Spatial2-seed#[985392D38082195A]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 116151 INFO  (SUITE-TestSolr4Spatial2-seed#[985392D38082195A]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1be0f3: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@2930a1
   [junit4]   2> 116152 INFO  (coreCloseExecutor-206-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@18bd1b9
   [junit4]   2> 116152 INFO  (coreCloseExecutor-206-thread-1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=18bd1b9
   [junit4]   2> 116152 INFO  (coreCloseExecutor-206-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1ec7fa0: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@ee4af3
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J1/temp/solr.search.TestSolr4Spatial2_985392D38082195A-003
   [junit4]   2> NOTE: test params are: codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=2, maxDocsPerChunk=539, blockSize=4), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=2, blockSize=4)), sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@e05e90), locale=sq-AL, timezone=Asia/Hovd
   [junit4]   2> NOTE: Linux 4.13.0-37-generic i386/Oracle Corporation 1.8.0_162 (32-bit)/cpus=8,threads=1,free=76378544,total=299368448
   [junit4]   2> NOTE: All tests run in this JVM: [MoveReplicaHDFSTest, TestSolr4Spatial2, TestSolr4Spatial2, TestSolr4Spatial2]
   [junit4] Completed [8/10 (5!)] on J1 in 0.38s, 16 tests, 1 error <<< FAILURES!

[...truncated 29 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/lucene/common-build.xml:1568: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/lucene/common-build.xml:1092: There were test failures: 10 suites, 100 tests, 5 errors [seed: 985392D38082195A]

Total time: 3 minutes 44 seconds

[repro] Setting last failure code to 256

[repro] Failures:
[repro]   0/5 failed: org.apache.solr.cloud.MoveReplicaHDFSTest
[repro]   5/5 failed: org.apache.solr.search.TestSolr4Spatial2
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=93f9a65b1c8aa460489fdce50ed84d18168b53ef, workspace=/var/lib/jenkins/workspace/Lucene-Solr-BadApples-master-Linux
[WARNINGS] Computing warning deltas based on reference build #21
Recording test results
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2

[JENKINS] Lucene-Solr-BadApples-master-Linux (32bit/jdk1.8.0_162) - Build # 21 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-BadApples-master-Linux/21/
Java: 32bit/jdk1.8.0_162 -server -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.cloud.TestAuthenticationFramework.testBasics

Error Message:
Error from server at https://127.0.0.1:45651/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/> <title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title> </head> <body><h2>HTTP ERROR 404</h2> <p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason: <pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.8.v20171121</a><hr/>  </body> </html> 

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at https://127.0.0.1:45651/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
<pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.8.v20171121</a><hr/>

</body>
</html>

	at __randomizedtesting.SeedInfo.seed([60110C3757D72F3E:5DC9A21B6F39714E]:0)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:551)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1015)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
	at org.apache.solr.cloud.TestAuthenticationFramework.collectionCreateSearchDeleteTwice(TestAuthenticationFramework.java:127)
	at org.apache.solr.cloud.TestAuthenticationFramework.testBasics(TestAuthenticationFramework.java:75)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:45651/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
<pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.8.v20171121</a><hr/>

</body>
</html>

	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:607)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:528)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	... 1 more




Build Log:
[...truncated 13553 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestAuthenticationFramework
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_60110C3757D72F3E-001/init-core-data-001
   [junit4]   2> 1653491 WARN  (SUITE-TestAuthenticationFramework-seed#[60110C3757D72F3E]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
   [junit4]   2> 1653491 INFO  (SUITE-TestAuthenticationFramework-seed#[60110C3757D72F3E]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1653493 INFO  (SUITE-TestAuthenticationFramework-seed#[60110C3757D72F3E]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 1653493 INFO  (SUITE-TestAuthenticationFramework-seed#[60110C3757D72F3E]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1653495 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[60110C3757D72F3E]) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_60110C3757D72F3E-001/tempDir-001
   [junit4]   2> 1653496 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[60110C3757D72F3E]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1653496 INFO  (Thread-5672) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1653496 INFO  (Thread-5672) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1653517 ERROR (Thread-5672) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1653596 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[60110C3757D72F3E]) [    ] o.a.s.c.ZkTestServer start zk server on port:33347
   [junit4]   2> 1653603 INFO  (zkConnectionManagerCallback-7762-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1653610 INFO  (jetty-launcher-7759-thread-1) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T09:27:37+12:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1653618 INFO  (jetty-launcher-7759-thread-2) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T09:27:37+12:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1653642 INFO  (jetty-launcher-7759-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1653643 INFO  (jetty-launcher-7759-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1653643 INFO  (jetty-launcher-7759-thread-2) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 1653643 INFO  (jetty-launcher-7759-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@cb429a{/solr,null,AVAILABLE}
   [junit4]   2> 1653644 INFO  (jetty-launcher-7759-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@15a3862{SSL,[ssl, http/1.1]}{127.0.0.1:45651}
   [junit4]   2> 1653644 INFO  (jetty-launcher-7759-thread-2) [    ] o.e.j.s.Server Started @1653686ms
   [junit4]   2> 1653644 INFO  (jetty-launcher-7759-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45651}
   [junit4]   2> 1653644 ERROR (jetty-launcher-7759-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1653644 INFO  (jetty-launcher-7759-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1653644 INFO  (jetty-launcher-7759-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 1653644 INFO  (jetty-launcher-7759-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1653644 INFO  (jetty-launcher-7759-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1653644 INFO  (jetty-launcher-7759-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-04-12T03:07:59.793Z
   [junit4]   2> 1653646 INFO  (jetty-launcher-7759-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1653646 INFO  (jetty-launcher-7759-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1653646 INFO  (jetty-launcher-7759-thread-1) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 1653647 INFO  (jetty-launcher-7759-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@16e160d{/solr,null,AVAILABLE}
   [junit4]   2> 1653647 INFO  (jetty-launcher-7759-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1b5229{SSL,[ssl, http/1.1]}{127.0.0.1:40355}
   [junit4]   2> 1653647 INFO  (jetty-launcher-7759-thread-1) [    ] o.e.j.s.Server Started @1653690ms
   [junit4]   2> 1653647 INFO  (jetty-launcher-7759-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40355}
   [junit4]   2> 1653647 ERROR (jetty-launcher-7759-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1653647 INFO  (jetty-launcher-7759-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1653647 INFO  (jetty-launcher-7759-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 1653647 INFO  (jetty-launcher-7759-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1653647 INFO  (jetty-launcher-7759-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1653647 INFO  (jetty-launcher-7759-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-04-12T03:07:59.796Z
   [junit4]   2> 1653656 INFO  (zkConnectionManagerCallback-7764-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1653656 INFO  (jetty-launcher-7759-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1653667 INFO  (zkConnectionManagerCallback-7766-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1653668 INFO  (jetty-launcher-7759-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1653675 INFO  (jetty-launcher-7759-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33347/solr
   [junit4]   2> 1653690 INFO  (zkConnectionManagerCallback-7770-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1653699 INFO  (jetty-launcher-7759-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33347/solr
   [junit4]   2> 1653701 INFO  (zkConnectionManagerCallback-7774-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1653701 INFO  (zkConnectionManagerCallback-7776-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1653716 INFO  (zkConnectionManagerCallback-7778-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1653725 INFO  (jetty-launcher-7759-thread-1) [n:127.0.0.1:40355_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1653726 INFO  (jetty-launcher-7759-thread-1) [n:127.0.0.1:40355_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40355_solr
   [junit4]   2> 1653726 INFO  (jetty-launcher-7759-thread-1) [n:127.0.0.1:40355_solr    ] o.a.s.c.Overseer Overseer (id=72124447647072262-127.0.0.1:40355_solr-n_0000000000) starting
   [junit4]   2> 1653744 INFO  (zkConnectionManagerCallback-7787-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1653746 INFO  (jetty-launcher-7759-thread-1) [n:127.0.0.1:40355_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33347/solr ready
   [junit4]   2> 1653752 DEBUG (OverseerStateUpdate-72124447647072262-127.0.0.1:40355_solr-n_0000000000) [    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1653752 INFO  (jetty-launcher-7759-thread-1) [n:127.0.0.1:40355_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40355_solr
   [junit4]   2> 1653756 INFO  (zkCallback-7777-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1653760 INFO  (zkCallback-7786-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1653761 INFO  (jetty-launcher-7759-thread-2) [n:127.0.0.1:45651_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1653762 INFO  (jetty-launcher-7759-thread-2) [n:127.0.0.1:45651_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1653763 INFO  (jetty-launcher-7759-thread-2) [n:127.0.0.1:45651_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1653763 INFO  (jetty-launcher-7759-thread-2) [n:127.0.0.1:45651_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45651_solr
   [junit4]   2> 1653768 INFO  (zkCallback-7775-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1653769 INFO  (zkCallback-7786-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1653772 INFO  (zkCallback-7777-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1653772 DEBUG (OverseerStateUpdate-72124447647072262-127.0.0.1:40355_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:45651_solr"} current state version: 0
   [junit4]   2> 1653826 INFO  (jetty-launcher-7759-thread-2) [n:127.0.0.1:45651_solr    ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin
   [junit4]   2> 1653867 INFO  (jetty-launcher-7759-thread-2) [n:127.0.0.1:45651_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45651.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14fd4d4
   [junit4]   2> 1653875 INFO  (jetty-launcher-7759-thread-2) [n:127.0.0.1:45651_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45651.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14fd4d4
   [junit4]   2> 1653875 INFO  (jetty-launcher-7759-thread-2) [n:127.0.0.1:45651_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45651.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14fd4d4
   [junit4]   2> 1653876 INFO  (jetty-launcher-7759-thread-2) [n:127.0.0.1:45651_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_60110C3757D72F3E-001/tempDir-001/node2/.
   [junit4]   2> 1653900 INFO  (zkConnectionManagerCallback-7792-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1653901 INFO  (jetty-launcher-7759-thread-2) [n:127.0.0.1:45651_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1653902 INFO  (jetty-launcher-7759-thread-2) [n:127.0.0.1:45651_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33347/solr ready
   [junit4]   2> 1655634 INFO  (jetty-launcher-7759-thread-1) [n:127.0.0.1:40355_solr    ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin
   [junit4]   2> 1655660 INFO  (jetty-launcher-7759-thread-1) [n:127.0.0.1:40355_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40355.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14fd4d4
   [junit4]   2> 1655668 INFO  (jetty-launcher-7759-thread-1) [n:127.0.0.1:40355_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40355.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14fd4d4
   [junit4]   2> 1655668 INFO  (jetty-launcher-7759-thread-1) [n:127.0.0.1:40355_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40355.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14fd4d4
   [junit4]   2> 1655670 INFO  (jetty-launcher-7759-thread-1) [n:127.0.0.1:40355_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_60110C3757D72F3E-001/tempDir-001/node1/.
   [junit4]   2> 1655686 INFO  (zkConnectionManagerCallback-7794-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1655689 INFO  (zkConnectionManagerCallback-7799-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1655690 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[60110C3757D72F3E]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1655691 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[60110C3757D72F3E]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33347/solr ready
   [junit4]   2> 1655692 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[60110C3757D72F3E]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testBasics
   [junit4]   2> 1655729 INFO  (qtp30393364-24016) [n:127.0.0.1:45651_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params async=615d52f1-bfae-4bce-b5cc-6d5f98e9d74a&replicationFactor=2&collection.configName=solrCloudCollectionConfig&maxShardsPerNode=2&name=testcollection&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1655730 INFO  (qtp30393364-24016) [n:127.0.0.1:45651_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=615d52f1-bfae-4bce-b5cc-6d5f98e9d74a&replicationFactor=2&collection.configName=solrCloudCollectionConfig&maxShardsPerNode=2&name=testcollection&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 1655731 INFO  (qtp30393364-24018) [n:127.0.0.1:45651_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=615d52f1-bfae-4bce-b5cc-6d5f98e9d74a&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1655731 INFO  (OverseerThreadFactory-6241-thread-1) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection testcollection
   [junit4]   2> 1655731 INFO  (qtp30393364-24018) [n:127.0.0.1:45651_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=615d52f1-bfae-4bce-b5cc-6d5f98e9d74a&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1655733 DEBUG (OverseerStateUpdate-72124447647072262-127.0.0.1:40355_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "name":"testcollection",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"2",
   [junit4]   2>   "collection.configName":"solrCloudCollectionConfig",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "maxShardsPerNode":"2",
   [junit4]   2>   "nrtReplicas":"2",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "async":"615d52f1-bfae-4bce-b5cc-6d5f98e9d74a",
   [junit4]   2>   "operation":"create"} current state version: 0
   [junit4]   2> 1655835 DEBUG (OverseerStateUpdate-72124447647072262-127.0.0.1:40355_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testcollection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:40355/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 1655835 INFO  (OverseerStateUpdate-72124447647072262-127.0.0.1:40355_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testcollection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:40355/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 1655836 DEBUG (OverseerStateUpdate-72124447647072262-127.0.0.1:40355_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testcollection_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:45651/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 1655836 INFO  (OverseerStateUpdate-72124447647072262-127.0.0.1:40355_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testcollection_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:45651/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 1655840 DEBUG (OverseerStateUpdate-72124447647072262-127.0.0.1:40355_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testcollection_shard2_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:40355/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 1655840 INFO  (OverseerStateUpdate-72124447647072262-127.0.0.1:40355_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testcollection_shard2_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:40355/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 1655841 DEBUG (OverseerStateUpdate-72124447647072262-127.0.0.1:40355_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 4, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testcollection_shard2_replica_n6",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:45651/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 1655841 INFO  (OverseerStateUpdate-72124447647072262-127.0.0.1:40355_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testcollection_shard2_replica_n6",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:45651/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 1656048 INFO  (qtp30393364-24014) [n:127.0.0.1:45651_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665&coreNodeName=core_node8&name=testcollection_shard2_replica_n6&action=CREATE&numShards=2&shard=shard2&wt=javabin} status=0 QTime=1
   [junit4]   2> 1656049 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665&coreNodeName=core_node8&name=testcollection_shard2_replica_n6&action=CREATE&numShards=2&shard=shard2&wt=javabin
   [junit4]   2> 1656049 INFO  (qtp30393364-24017) [n:127.0.0.1:45651_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503&coreNodeName=core_node5&name=testcollection_shard1_replica_n2&action=CREATE&numShards=2&shard=shard1&wt=javabin} status=0 QTime=1
   [junit4]   2> 1656049 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503&coreNodeName=core_node5&name=testcollection_shard1_replica_n2&action=CREATE&numShards=2&shard=shard1&wt=javabin
   [junit4]   2> 1656051 DEBUG (OverseerStateUpdate-72124447647072262-127.0.0.1:40355_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"testcollection_shard1_replica_n2",
   [junit4]   2>   "core_node_name":"core_node5",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:45651/solr",
   [junit4]   2>   "node_name":"127.0.0.1:45651_solr",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1656059 DEBUG (OverseerStateUpdate-72124447647072262-127.0.0.1:40355_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "core":"testcollection_shard2_replica_n6",
   [junit4]   2>   "core_node_name":"core_node8",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:45651/solr",
   [junit4]   2>   "node_name":"127.0.0.1:45651_solr",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1656078 INFO  (qtp8016180-24086) [n:127.0.0.1:40355_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107454312983&coreNodeName=core_node3&name=testcollection_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&wt=javabin} status=0 QTime=0
   [junit4]   2> 1656078 INFO  (parallelCoreAdminExecutor-6244-thread-1-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107454312983 CREATE) [n:127.0.0.1:40355_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107454312983&coreNodeName=core_node3&name=testcollection_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&wt=javabin
   [junit4]   2> 1656080 INFO  (parallelCoreAdminExecutor-6244-thread-1-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107454312983 CREATE) [n:127.0.0.1:40355_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1656082 INFO  (qtp8016180-24085) [n:127.0.0.1:40355_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107456426724&coreNodeName=core_node7&name=testcollection_shard2_replica_n4&action=CREATE&numShards=2&shard=shard2&wt=javabin} status=0 QTime=1
   [junit4]   2> 1656084 INFO  (parallelCoreAdminExecutor-6244-thread-2-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107456426724 CREATE) [n:127.0.0.1:40355_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107456426724&coreNodeName=core_node7&name=testcollection_shard2_replica_n4&action=CREATE&numShards=2&shard=shard2&wt=javabin
   [junit4]   2> 1656084 INFO  (qtp30393364-24013) [n:127.0.0.1:45651_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665
   [junit4]   2> 1656084 INFO  (qtp30393364-24013) [n:127.0.0.1:45651_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1656087 DEBUG (OverseerStateUpdate-72124447647072262-127.0.0.1:40355_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
   [junit4]   2>   "core":"testcollection_shard1_replica_n1",
   [junit4]   2>   "core_node_name":"core_node3",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:40355/solr",
   [junit4]   2>   "node_name":"127.0.0.1:40355_solr",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1656087 DEBUG (OverseerStateUpdate-72124447647072262-127.0.0.1:40355_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 4, message = {
   [junit4]   2>   "core":"testcollection_shard2_replica_n4",
   [junit4]   2>   "core_node_name":"core_node7",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:40355/solr",
   [junit4]   2>   "node_name":"127.0.0.1:40355_solr",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1656188 INFO  (zkCallback-7777-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1656188 INFO  (zkCallback-7775-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1656188 INFO  (zkCallback-7777-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1656188 INFO  (zkCallback-7775-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1656188 INFO  (zkCallback-7777-thread-3) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1656732 INFO  (qtp30393364-24016) [n:127.0.0.1:45651_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=615d52f1-bfae-4bce-b5cc-6d5f98e9d74a&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1656733 INFO  (qtp30393364-24016) [n:127.0.0.1:45651_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=615d52f1-bfae-4bce-b5cc-6d5f98e9d74a&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1657057 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1657061 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1657062 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.s.IndexSchema [testcollection_shard1_replica_n2] Schema name=minimal
   [junit4]   2> 1657064 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1657064 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard1_replica_n2' using configuration from collection testcollection, trusted=true
   [junit4]   2> 1657064 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45651.solr.core.testcollection.shard1.replica_n2' (registry 'solr.core.testcollection.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14fd4d4
   [junit4]   2> 1657064 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1657065 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.SolrCore [[testcollection_shard1_replica_n2] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_60110C3757D72F3E-001/tempDir-001/node2/testcollection_shard1_replica_n2], dataDir=[/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_60110C3757D72F3E-001/tempDir-001/node2/./testcollection_shard1_replica_n2/data/]
   [junit4]   2> 1657065 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.s.IndexSchema [testcollection_shard2_replica_n6] Schema name=minimal
   [junit4]   2> 1657066 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1657066 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard2_replica_n6' using configuration from collection testcollection, trusted=true
   [junit4]   2> 1657066 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45651.solr.core.testcollection.shard2.replica_n6' (registry 'solr.core.testcollection.shard2.replica_n6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14fd4d4
   [junit4]   2> 1657067 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1657067 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.SolrCore [[testcollection_shard2_replica_n6] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_60110C3757D72F3E-001/tempDir-001/node2/testcollection_shard2_replica_n6], dataDir=[/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_60110C3757D72F3E-001/tempDir-001/node2/./testcollection_shard2_replica_n6/data/]
   [junit4]   2> 1657086 INFO  (qtp30393364-24091) [n:127.0.0.1:45651_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665
   [junit4]   2> 1657086 INFO  (qtp30393364-24091) [n:127.0.0.1:45651_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1657108 INFO  (parallelCoreAdminExecutor-6244-thread-2-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107456426724 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1657108 INFO  (parallelCoreAdminExecutor-6244-thread-1-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107454312983 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1657114 INFO  (parallelCoreAdminExecutor-6244-thread-1-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107454312983 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.s.IndexSchema [testcollection_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 1657114 INFO  (parallelCoreAdminExecutor-6244-thread-2-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107456426724 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.s.IndexSchema [testcollection_shard2_replica_n4] Schema name=minimal
   [junit4]   2> 1657116 INFO  (parallelCoreAdminExecutor-6244-thread-2-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107456426724 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1657116 INFO  (parallelCoreAdminExecutor-6244-thread-1-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107454312983 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1657116 INFO  (parallelCoreAdminExecutor-6244-thread-2-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107456426724 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard2_replica_n4' using configuration from collection testcollection, trusted=true
   [junit4]   2> 1657116 INFO  (parallelCoreAdminExecutor-6244-thread-1-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107454312983 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard1_replica_n1' using configuration from collection testcollection, trusted=true
   [junit4]   2> 1657116 INFO  (parallelCoreAdminExecutor-6244-thread-1-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107454312983 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40355.solr.core.testcollection.shard1.replica_n1' (registry 'solr.core.testcollection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14fd4d4
   [junit4]   2> 1657116 INFO  (parallelCoreAdminExecutor-6244-thread-2-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107456426724 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40355.solr.core.testcollection.shard2.replica_n4' (registry 'solr.core.testcollection.shard2.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14fd4d4
   [junit4]   2> 1657116 INFO  (parallelCoreAdminExecutor-6244-thread-2-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107456426724 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1657116 INFO  (parallelCoreAdminExecutor-6244-thread-1-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107454312983 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1657116 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1657116 INFO  (parallelCoreAdminExecutor-6244-thread-2-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107456426724 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.SolrCore [[testcollection_shard2_replica_n4] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_60110C3757D72F3E-001/tempDir-001/node1/testcollection_shard2_replica_n4], dataDir=[/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_60110C3757D72F3E-001/tempDir-001/node1/./testcollection_shard2_replica_n4/data/]
   [junit4]   2> 1657116 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1657116 INFO  (parallelCoreAdminExecutor-6244-thread-1-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107454312983 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.SolrCore [[testcollection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_60110C3757D72F3E-001/tempDir-001/node1/testcollection_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_60110C3757D72F3E-001/tempDir-001/node1/./testcollection_shard1_replica_n1/data/]
   [junit4]   2> 1657117 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1657117 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1657118 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.s.SolrIndexSearcher Opening [Searcher@a6f56a[testcollection_shard2_replica_n6] main]
   [junit4]   2> 1657118 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
   [junit4]   2> 1657119 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 1657119 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1657120 INFO  (searcherExecutor-6247-thread-1-processing-x:testcollection_shard2_replica_n6 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 r:core_node8 CREATE n:127.0.0.1:45651_solr c:testcollection s:shard2) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.SolrCore [testcollection_shard2_replica_n6] Registered new searcher Searcher@a6f56a[testcollection_shard2_replica_n6] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1657120 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1597508139896274944
   [junit4]   2> 1657124 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testcollection/terms/shard2 to Terms{values={core_node8=0}, version=0}
   [junit4]   2> 1657124 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1657124 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1657126 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1657126 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1657127 DEBUG (OverseerStateUpdate-72124447647072262-127.0.0.1:40355_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "collection":"testcollection"} current state version: 0
   [junit4]   2> 1657127 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@23a87e[testcollection_shard1_replica_n2] main]
   [junit4]   2> 1657127 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoutin=9999ms
   [junit4]   2> 1657128 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
   [junit4]   2> 1657128 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 1657128 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1657129 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1597508139905712128
   [junit4]   2> 1657131 INFO  (searcherExecutor-6246-thread-1-processing-x:testcollection_shard1_replica_n2 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 r:core_node5 CREATE n:127.0.0.1:45651_solr c:testcollection s:shard1) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.SolrCore [testcollection_shard1_replica_n2] Registered new searcher Searcher@23a87e[testcollection_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1657131 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testcollection/terms/shard1 to Terms{values={core_node5=0}, version=0}
   [junit4]   2> 1657133 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
   [junit4]   2> 1657133 DEBUG (OverseerStateUpdate-72124447647072262-127.0.0.1:40355_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testcollection"} current state version: 0
   [junit4]   2> 1657152 INFO  (parallelCoreAdminExecutor-6244-thread-1-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107454312983 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1657152 INFO  (parallelCoreAdminExecutor-6244-thread-1-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107454312983 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1657154 INFO  (parallelCoreAdminExecutor-6244-thread-1-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107454312983 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1657154 INFO  (parallelCoreAdminExecutor-6244-thread-1-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107454312983 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1657155 INFO  (parallelCoreAdminExecutor-6244-thread-1-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107454312983 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@32549b[testcollection_shard1_replica_n1] main]
   [junit4]   2> 1657156 INFO  (parallelCoreAdminExecutor-6244-thread-1-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107454312983 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
   [junit4]   2> 1657156 INFO  (parallelCoreAdminExecutor-6244-thread-1-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107454312983 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 1657156 INFO  (parallelCoreAdminExecutor-6244-thread-2-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107456426724 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1657156 INFO  (parallelCoreAdminExecutor-6244-thread-2-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107456426724 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1657156 INFO  (parallelCoreAdminExecutor-6244-thread-1-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107454312983 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1657157 INFO  (searcherExecutor-6251-thread-1-processing-x:testcollection_shard1_replica_n1 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107454312983 r:core_node3 CREATE n:127.0.0.1:40355_solr c:testcollection s:shard1) [n:127.0.0.1:40355_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.SolrCore [testcollection_shard1_replica_n1] Registered new searcher Searcher@32549b[testcollection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1657157 INFO  (parallelCoreAdminExecutor-6244-thread-1-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107454312983 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1597508139935072256
   [junit4]   2> 1657157 INFO  (parallelCoreAdminExecutor-6244-thread-2-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107456426724 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1657157 INFO  (parallelCoreAdminExecutor-6244-thread-2-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107456426724 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1657159 INFO  (parallelCoreAdminExecutor-6244-thread-2-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107456426724 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.s.SolrIndexSearcher Opening [Searcher@10af0b1[testcollection_shard2_replica_n4] main]
   [junit4]   2> 1657159 INFO  (parallelCoreAdminExecutor-6244-thread-2-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107456426724 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
   [junit4]   2> 1657159 INFO  (parallelCoreAdminExecutor-6244-thread-2-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107456426724 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 1657159 INFO  (parallelCoreAdminExecutor-6244-thread-1-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107454312983 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testcollection/terms/shard1 to Terms{values={core_node3=0, core_node5=0}, version=1}
   [junit4]   2> 1657160 INFO  (parallelCoreAdminExecutor-6244-thread-2-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107456426724 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1657161 INFO  (searcherExecutor-6250-thread-1-processing-x:testcollection_shard2_replica_n4 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107456426724 r:core_node7 CREATE n:127.0.0.1:40355_solr c:testcollection s:shard2) [n:127.0.0.1:40355_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.SolrCore [testcollection_shard2_replica_n4] Registered new searcher Searcher@10af0b1[testcollection_shard2_replica_n4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1657161 INFO  (parallelCoreAdminExecutor-6244-thread-2-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107456426724 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1597508139939266560
   [junit4]   2> 1657163 INFO  (parallelCoreAdminExecutor-6244-thread-2-processing-n:127.0.0.1:40355_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107456426724 CREATE) [n:127.0.0.1:40355_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testcollection/terms/shard2 to Terms{values={core_node7=0, core_node8=0}, version=1}
   [junit4]   2> 1657234 INFO  (zkCallback-7775-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1657234 INFO  (zkCallback-7777-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1657234 INFO  (zkCallback-7775-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1657234 INFO  (zkCallback-7777-thread-4) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1657234 INFO  (zkCallback-7777-thread-3) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1657628 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1657628 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1657628 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:45651/solr/testcollection_shard2_replica_n6/
   [junit4]   2> 1657628 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.PeerSync PeerSync: core=testcollection_shard2_replica_n6 url=https://127.0.0.1:45651/solr START replicas=[https://127.0.0.1:40355/solr/testcollection_shard2_replica_n4/] nUpdates=100
   [junit4]   2> 1657631 INFO  (qtp8016180-24085) [n:127.0.0.1:40355_solr c:testcollection s:shard2 r:core_node7 x:testcollection_shard2_replica_n4] o.a.s.c.S.Request [testcollection_shard2_replica_n4]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1657633 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1657634 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1657634 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:45651/solr/testcollection_shard1_replica_n2/
   [junit4]   2> 1657634 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=testcollection_shard1_replica_n2 url=https://127.0.0.1:45651/solr START replicas=[https://127.0.0.1:40355/solr/testcollection_shard1_replica_n1/] nUpdates=100
   [junit4]   2> 1657635 INFO  (qtp8016180-24086) [n:127.0.0.1:40355_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica_n1] o.a.s.c.S.Request [testcollection_shard1_replica_n1]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1657733 INFO  (qtp30393364-24013) [n:127.0.0.1:45651_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=615d52f1-bfae-4bce-b5cc-6d5f98e9d74a&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1657734 INFO  (qtp30393364-24013) [n:127.0.0.1:45651_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=615d52f1-bfae-4bce-b5cc-6d5f98e9d74a&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1657928 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.u.PeerSync PeerSync: core=testcollection_shard2_replica_n6 url=https://127.0.0.1:45651/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 1657928 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 1657929 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 1657929 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1657930 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:45651/solr/testcollection_shard2_replica_n6/ shard2
   [junit4]   2> 1657931 DEBUG (OverseerStateUpdate-72124447647072262-127.0.0.1:40355_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "base_url":"https://127.0.0.1:45651/solr",
   [junit4]   2>   "core":"testcollection_shard2_replica_n6",
   [junit4]   2>   "state":"active"} current state version: 0
   [junit4]   2> 1657934 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=testcollection_shard1_replica_n2 url=https://127.0.0.1:45651/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 1657934 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 1657934 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 1657934 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1657936 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:45651/solr/testcollection_shard1_replica_n2/ shard1
   [junit4]   2> 1657936 DEBUG (OverseerStateUpdate-72124447647072262-127.0.0.1:40355_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "base_url":"https://127.0.0.1:45651/solr",
   [junit4]   2>   "core":"testcollection_shard1_replica_n2",
   [junit4]   2>   "state":"active"} current state version: 0
   [junit4]   2> 1658038 INFO  (zkCallback-7775-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1658038 INFO  (zkCallback-7775-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1658038 INFO  (zkCallback-7777-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1658038 INFO  (zkCallback-7777-thread-4) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1658038 INFO  (zkCallback-7777-thread-3) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1658081 INFO  (parallelCoreAdminExecutor-6242-thread-1-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n6] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1658086 INFO  (parallelCoreAdminExecutor-6242-thread-2-processing-n:127.0.0.1:45651_solr 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107455343503 CREATE) [n:127.0.0.1:45651_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1658087 INFO  (qtp30393364-24016) [n:127.0.0.1:45651_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665
   [junit4]   2> 1658087 INFO  (qtp30393364-24016) [n:127.0.0.1:45651_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1658087 DEBUG (OverseerStateUpdate-72124447647072262-127.0.0.1:40355_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"testcollection_shard2_replica_n6",
   [junit4]   2>   "core_node_name":"core_node8",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:45651/solr",
   [junit4]   2>   "node_name":"127.0.0.1:45651_solr",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1658088 DEBUG (OverseerStateUpdate-72124447647072262-127.0.0.1:40355_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "core":"testcollection_shard1_replica_n2",
   [junit4]   2>   "core_node_name":"core_node5",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:45651/solr",
   [junit4]   2>   "node_name":"127.0.0.1:45651_solr",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1658162 DEBUG (OverseerStateUpdate-72124447647072262-127.0.0.1:40355_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
   [junit4]   2>   "core":"testcollection_shard1_replica_n1",
   [junit4]   2>   "core_node_name":"core_node3",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:40355/solr",
   [junit4]   2>   "node_name":"127.0.0.1:40355_solr",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1658165 DEBUG (OverseerStateUpdate-72124447647072262-127.0.0.1:40355_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 4, message = {
   [junit4]   2>   "core":"testcollection_shard2_replica_n4",
   [junit4]   2>   "core_node_name":"core_node7",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:40355/solr",
   [junit4]   2>   "node_name":"127.0.0.1:40355_solr",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1658266 INFO  (zkCallback-7775-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1658266 INFO  (zkCallback-7775-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1658267 INFO  (zkCallback-7777-thread-3) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1658267 INFO  (zkCallback-7777-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1658267 INFO  (zkCallback-7777-thread-4) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1658737 INFO  (qtp30393364-24091) [n:127.0.0.1:45651_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=615d52f1-bfae-4bce-b5cc-6d5f98e9d74a&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1658752 INFO  (qtp30393364-24091) [n:127.0.0.1:45651_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=615d52f1-bfae-4bce-b5cc-6d5f98e9d74a&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=15
   [junit4]   2> 1659088 INFO  (qtp30393364-24013) [n:127.0.0.1:45651_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665
   [junit4]   2> 1659088 INFO  (qtp30393364-24013) [n:127.0.0.1:45651_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107460329665&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1659091 INFO  (qtp8016180-24022) [n:127.0.0.1:40355_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107456426724
   [junit4]   2> 1659091 INFO  (qtp8016180-24022) [n:127.0.0.1:40355_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=615d52f1-bfae-4bce-b5cc-6d5f98e9d74a1020107456426724&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1659094 INFO  (OverseerThreadFactory-6241-thread-1) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1659732 INFO  (OverseerCollectionConfigSetProcessor-72124447647072262-127.0.0.1:40355_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1659753 INFO  (qtp30393364-24016) [n:127.0.0.1:45651_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=615d52f1-bfae-4bce-b5cc-6d5f98e9d74a&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1659754 INFO  (qtp30393364-24016) [n:127.0.0.1:45651_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=615d52f1-bfae-4bce-b5cc-6d5f98e9d74a&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1659755 INFO  (qtp30393364-24013) [n:127.0.0.1:45651_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletestatus with params requestid=615d52f1-bfae-4bce-b5cc-6d5f98e9d74a&action=DELETESTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1659755 INFO  (qtp30393364-24013) [n:127.0.0.1:45651_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=615d52f1-bfae-4bce-b5cc-6d5f98e9d74a&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1659755 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[60110C3757D72F3E]) [    ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: testcollection failOnTimeout:true timeout (sec):330
   [junit4]   1> -
   [junit4]   1> replica:core_node3 rstate:active live:true
   [junit4]   1>

[...truncated too long message...]

ica_n7, tag=f286c4
   [junit4]   2> 1664606 INFO  (coreCloseExecutor-6286-thread-2) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@edda1f: rootName = solr_45651, domain = solr.core.testcollection.shard2.replica_n7, service url = null, agent id = null] for registry solr.core.testcollection.shard2.replica_n7 / com.codahale.metrics.MetricRegistry@1d80d22
   [junit4]   2> 1664609 INFO  (coreCloseExecutor-6287-thread-2) [n:127.0.0.1:40355_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testcollection.shard1.replica_n1, tag=a80727
   [junit4]   2> 1664609 INFO  (coreCloseExecutor-6287-thread-2) [n:127.0.0.1:40355_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@f75134: rootName = solr_40355, domain = solr.core.testcollection.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.testcollection.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@1479c40
   [junit4]   2> 1664613 INFO  (coreCloseExecutor-6286-thread-2) [n:127.0.0.1:45651_solr c:testcollection s:shard2 r:core_node8 x:testcollection_shard2_replica_n7] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testcollection.shard2.leader, tag=f286c4
   [junit4]   2> 1664616 INFO  (jetty-closer-7760-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@cb429a{/solr,null,UNAVAILABLE}
   [junit4]   2> 1664616 INFO  (jetty-closer-7760-thread-1) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 1664620 INFO  (coreCloseExecutor-6287-thread-2) [n:127.0.0.1:40355_solr c:testcollection s:shard1 r:core_node2 x:testcollection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testcollection.shard1.leader, tag=a80727
   [junit4]   2> 1664620 INFO  (coreCloseExecutor-6287-thread-1) [n:127.0.0.1:40355_solr c:testcollection s:shard2 r:core_node6 x:testcollection_shard2_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testcollection.shard2.leader, tag=dc8958
   [junit4]   2> 1664622 INFO  (jetty-closer-7760-thread-2) [    ] o.a.s.c.Overseer Overseer (id=72124447647072262-127.0.0.1:40355_solr-n_0000000000) closing
   [junit4]   2> 1664622 INFO  (OverseerStateUpdate-72124447647072262-127.0.0.1:40355_solr-n_0000000000) [    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:40355_solr
   [junit4]   2> 1664622 WARN  (OverseerAutoScalingTriggerThread-72124447647072262-127.0.0.1:40355_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 1664628 INFO  (jetty-closer-7760-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@16e160d{/solr,null,UNAVAILABLE}
   [junit4]   2> 1664628 INFO  (jetty-closer-7760-thread-2) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 1664631 ERROR (SUITE-TestAuthenticationFramework-seed#[60110C3757D72F3E]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1664631 INFO  (SUITE-TestAuthenticationFramework-seed#[60110C3757D72F3E]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:33347 33347
   [junit4]   2> 1664746 INFO  (Thread-5672) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:33347 33347
   [junit4]   2> 1664746 WARN  (Thread-5672) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	14	/solr/collections/testcollection/terms/shard1
   [junit4]   2> 	12	/solr/collections/testcollection/terms/shard2
   [junit4]   2> 	5	/solr/aliases.json
   [junit4]   2> 	4	/solr/configs/solrCloudCollectionConfig
   [junit4]   2> 	2	/solr/security.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	24	/solr/collections/testcollection/state.json
   [junit4]   2> 	5	/solr/clusterstate.json
   [junit4]   2> 	5	/solr/clusterprops.json
   [junit4]   2> 	2	/solr/collections/testcollection/leader_elect/shard1/election/72124447647072261-core_node5-n_0000000000
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	5	/solr/live_nodes
   [junit4]   2> 	5	/solr/collections
   [junit4]   2> 
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-BadApples-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestAuthenticationFramework_60110C3757D72F3E-001
   [junit4]   2> Apr 12, 2018 3:08:10 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {id=BlockTreeOrds(blocksize=128)}, docValues:{_version_=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=166, maxMBSortInHeap=6.453623215252316, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@9a8fa1), locale=es-UY, timezone=Europe/Helsinki
   [junit4]   2> NOTE: Linux 4.13.0-37-generic i386/Oracle Corporation 1.8.0_162 (32-bit)/cpus=8,threads=1,free=287453160,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [TestFreeTextSuggestions, TestRandomCollapseQParserPlugin, LIROnShardRestartTest, TestRandomDVFaceting, CdcrReplicationDistributedZkTest, HdfsRestartWhileUpdatingTest, ShardSplitTest, TestUnInvertedFieldException, NodeLostTriggerTest, OverseerTaskQueueTest, TestManagedSynonymGraphFilterFactory, ConcurrentCreateRoutedAliasTest, TestConfigsApi, TestScoreJoinQPNoScore, TestNumericTerms32, MultiSolrCloudTestCaseTest, DeleteLastCustomShardedReplicaTest, SolrCmdDistributorTest, TestTestInjection, TestSha256AuthenticationProvider, TestReplicaProperties, IndexSizeTriggerTest, HdfsNNFailoverTest, CdcrVersionReplicationTest, TestRecoveryHdfs, ResponseHeaderTest, ExecutePlanActionTest, TestSkipOverseerOperations, TestDFRSimilarityFactory, MinimalSchemaTest, DateFieldTest, LeaderElectionIntegrationTest, DistributedQueryComponentOptimizationTest, HdfsLockFactoryTest, SpatialFilterTest, ResponseLogComponentTest, CoreAdminHandlerTest, TestStressLiveNodes, TestClusterProperties, ClusterStateUpdateTest, TestPivotHelperCode, TestStressUserVersions, TestManagedSchema, NotRequiredUniqueKeyTest, BitVectorTest, CursorMarkTest, NoCacheHeaderTest, TestPullReplicaErrorHandling, DistributedFacetPivotLargeTest, BasicDistributedZk2Test, ConvertedLegacyTest, AutoAddReplicasPlanActionTest, HdfsBasicDistributedZkTest, ReturnFieldsTest, TestSimpleQParserPlugin, TestLFUCache, LukeRequestHandlerTest, TestNumericRangeQuery64, TestCorePropertiesReload, LegacyCloudClusterPropTest, AddReplicaTest, TestSolrConfigHandlerCloud, TestRealTimeGet, TestAnalyzeInfixSuggestions, SolrCLIZkUtilsTest, TestRebalanceLeaders, ZkStateWriterTest, FileBasedSpellCheckerTest, TestXmlQParserPlugin, TestDistribDocBasedVersion, TestDistribStateManager, TestInitParams, TestPayloadCheckQParserPlugin, TestDocBasedVersionConstraints, TestSchemaSimilarityResource, InfixSuggestersTest, SearchRateTriggerIntegrationTest, TestCodecSupport, TestCopyFieldCollectionResource, TestBadConfig, WordBreakSolrSpellCheckerTest, NumericFieldsTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestGraphMLResponseWriter, TestSolrCloudWithSecureImpersonation, PreAnalyzedFieldManagedSchemaCloudTest, ClusterStateTest, HLLSerializationTest, TestInPlaceUpdatesDistrib, DocValuesTest, DateMathParserTest, TestPointFields, IgnoreCommitOptimizeUpdateProcessorFactoryTest, StatsReloadRaceTest, GraphQueryTest, CacheHeaderTest, TestLazyCores, TestLMJelinekMercerSimilarityFactory, TestSuggestSpellingConverter, TestQueryTypes, TestSortableTextField, NodeAddedTriggerIntegrationTest, TestRawResponseWriter, TestDistributedMissingSort, MoreLikeThisHandlerTest, DeleteShardTest, PingRequestHandlerTest, TestUniqueKeyFieldResource, TestLegacyNumericRangeQueryBuilder, DistribDocExpirationUpdateProcessorTest, TestLegacyTerms, CoreMergeIndexesAdminHandlerTest, TestSegmentSorting, TestDistributedStatsComponentCardinality, CreateCollectionCleanupTest, DocValuesMultiTest, TriLevelCompositeIdRoutingTest, BasicAuthIntegrationTest, LeaderElectionTest, DirectSolrSpellCheckerTest, TestAuthenticationFramework]
   [junit4] Completed [444/802 (1!)] on J2 in 12.03s, 1 test, 1 error <<< FAILURES!

[...truncated 46451 lines...]
[repro] Jenkins log URL: https://jenkins.thetaphi.de/job/Lucene-Solr-BadApples-master-Linux/21/consoleText

[repro] Revision: 8927d469cb05c27db6864076010de039302c7e25

[repro] Repro line:  ant test  -Dtestcase=TestAuthenticationFramework -Dtests.method=testBasics -Dtests.seed=60110C3757D72F3E -Dtests.multiplier=3 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=es-UY -Dtests.timezone=Europe/Helsinki -Dtests.asserts=true -Dtests.file.encoding=US-ASCII

[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]    solr/core
[repro]       TestAuthenticationFramework
[repro] ant compile-test

[...truncated 3294 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.TestAuthenticationFramework" -Dtests.showOutput=onerror  -Dtests.seed=60110C3757D72F3E -Dtests.multiplier=3 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=es-UY -Dtests.timezone=Europe/Helsinki -Dtests.asserts=true -Dtests.file.encoding=US-ASCII

[...truncated 71 lines...]
[repro] Failures:
[repro]   0/5 failed: org.apache.solr.cloud.TestAuthenticationFramework
[repro] Exiting with code 0

[...truncated 40 lines...]