You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2018/04/03 11:27:58 UTC

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1779 - Unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1779/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testSelectedCollections

Error Message:
Timed out waiting for replicas of new collection to be active null Live Nodes: [127.0.0.1:33831_solr, 127.0.0.1:46039_solr, 127.0.0.1:62664_solr, 127.0.0.1:65471_solr] Last available state: null

Stack Trace:
java.lang.AssertionError: Timed out waiting for replicas of new collection to be active
null
Live Nodes: [127.0.0.1:33831_solr, 127.0.0.1:46039_solr, 127.0.0.1:62664_solr, 127.0.0.1:65471_solr]
Last available state: null
	at __randomizedtesting.SeedInfo.seed([64DEE8598928EE50:5E700D80B74C373E]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:269)
	at org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testSelectedCollections(ComputePlanActionTest.java:451)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 1851 lines...]
   [junit4] JVM J1: stdout was not empty, see: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/build/core/test/temp/junit4-J1-20180403_094202_2027697162946924646552.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] codec: CheapBastard, pf: Lucene50, dvf: Direct
   [junit4] <<< JVM J1: EOF ----

[...truncated 12225 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.ComputePlanActionTest
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.ComputePlanActionTest_64DEE8598928EE50-001/init-core-data-001
   [junit4]   2> 2695932 INFO  (SUITE-ComputePlanActionTest-seed#[64DEE8598928EE50]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.ComputePlanActionTest_64DEE8598928EE50-001/tempDir-001
   [junit4]   2> 2695932 INFO  (SUITE-ComputePlanActionTest-seed#[64DEE8598928EE50]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2695933 INFO  (Thread-6350) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2695933 INFO  (Thread-6350) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2695936 ERROR (Thread-6350) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 2696033 INFO  (SUITE-ComputePlanActionTest-seed#[64DEE8598928EE50]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:64060
   [junit4]   2> 2696040 INFO  (zkConnectionManagerCallback-8937-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2696063 INFO  (jetty-launcher-8934-thread-1) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 2696064 INFO  (jetty-launcher-8934-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 2696064 INFO  (jetty-launcher-8934-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 2696064 INFO  (jetty-launcher-8934-thread-1) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 2696066 INFO  (jetty-launcher-8934-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3aaf8a51{/solr,null,AVAILABLE}
   [junit4]   2> 2696066 INFO  (jetty-launcher-8934-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@56ef9d3e{HTTP/1.1,[http/1.1]}{127.0.0.1:51624}
   [junit4]   2> 2696068 INFO  (jetty-launcher-8934-thread-1) [    ] o.e.j.s.Server Started @2697812ms
   [junit4]   2> 2696068 INFO  (jetty-launcher-8934-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=51624}
   [junit4]   2> 2696068 ERROR (jetty-launcher-8934-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2696068 INFO  (jetty-launcher-8934-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 2696068 INFO  (jetty-launcher-8934-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 2696068 INFO  (jetty-launcher-8934-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2696068 INFO  (jetty-launcher-8934-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2696068 INFO  (jetty-launcher-8934-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-04-03T10:49:59.030Z
   [junit4]   2> 2696071 INFO  (zkConnectionManagerCallback-8939-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2696072 INFO  (jetty-launcher-8934-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2696083 INFO  (jetty-launcher-8934-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:64060/solr
   [junit4]   2> 2696085 INFO  (zkConnectionManagerCallback-8943-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2696089 INFO  (zkConnectionManagerCallback-8945-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2696302 INFO  (jetty-launcher-8934-thread-1) [n:127.0.0.1:51624_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2696303 INFO  (jetty-launcher-8934-thread-1) [n:127.0.0.1:51624_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51624_solr
   [junit4]   2> 2696304 INFO  (jetty-launcher-8934-thread-1) [n:127.0.0.1:51624_solr    ] o.a.s.c.Overseer Overseer (id=72170970746585091-127.0.0.1:51624_solr-n_0000000000) starting
   [junit4]   2> 2696317 INFO  (zkConnectionManagerCallback-8952-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2696321 INFO  (jetty-launcher-8934-thread-1) [n:127.0.0.1:51624_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:64060/solr ready
   [junit4]   2> 2696324 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 2696324 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 2696325 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: []
   [junit4]   2> 2696325 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 2696325 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 2696331 INFO  (jetty-launcher-8934-thread-1) [n:127.0.0.1:51624_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51624_solr
   [junit4]   2> 2696331 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 2696334 INFO  (zkCallback-8951-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2696335 INFO  (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2696342 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 2696342 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 2696342 DEBUG (ScheduledTrigger-9945-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 2697367 DEBUG (ScheduledTrigger-9945-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 2697908 INFO  (jetty-launcher-8934-thread-1) [n:127.0.0.1:51624_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51624.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ce44515
   [junit4]   2> 2697917 INFO  (jetty-launcher-8934-thread-1) [n:127.0.0.1:51624_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51624.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ce44515
   [junit4]   2> 2697917 INFO  (jetty-launcher-8934-thread-1) [n:127.0.0.1:51624_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51624.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ce44515
   [junit4]   2> 2697919 INFO  (jetty-launcher-8934-thread-1) [n:127.0.0.1:51624_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.ComputePlanActionTest_64DEE8598928EE50-001/tempDir-001/node1/.
   [junit4]   2> 2697944 INFO  (zkConnectionManagerCallback-8954-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2697949 INFO  (zkConnectionManagerCallback-8959-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2697999 INFO  (SUITE-ComputePlanActionTest-seed#[64DEE8598928EE50]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2698002 INFO  (SUITE-ComputePlanActionTest-seed#[64DEE8598928EE50]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:64060/solr ready
   [junit4]   2> 2698214 DEBUG (zkCallback-8951-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 2698214 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 2698215 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2698222 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 2698222 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 2698222 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 2698245 INFO  (qtp33066169-30773) [n:127.0.0.1:51624_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 2698245 DEBUG (qtp33066169-30777) [n:127.0.0.1:51624_solr    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 2698246 INFO  (qtp33066169-30777) [n:127.0.0.1:51624_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=8
   [junit4]   2> 2698246 DEBUG (zkCallback-8951-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 2698246 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 2698246 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 2698247 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 2698247 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 2698249 INFO  (qtp33066169-30773) [n:127.0.0.1:51624_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 2698249 DEBUG (qtp33066169-30776) [n:127.0.0.1:51624_solr    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 2698250 INFO  (qtp33066169-30776) [n:127.0.0.1:51624_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=3
   [junit4]   2> 2698250 DEBUG (zkCallback-8951-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 2698250 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 2698250 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 2698251 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 2698252 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 2698265 INFO  (qtp33066169-30776) [n:127.0.0.1:51624_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 2698265 DEBUG (qtp33066169-30774) [n:127.0.0.1:51624_solr    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 2698266 INFO  (qtp33066169-30774) [n:127.0.0.1:51624_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=4
   [junit4]   2> 2698268 DEBUG (zkCallback-8951-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
   [junit4]   2> 2698269 INFO  (qtp33066169-30774) [n:127.0.0.1:51624_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 2698269 DEBUG (qtp33066169-30773) [n:127.0.0.1:51624_solr    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 2698272 INFO  (qtp33066169-30773) [n:127.0.0.1:51624_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=6
   [junit4]   2> 2698273 INFO  (qtp33066169-30776) [n:127.0.0.1:51624_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&name=testNodeAdded&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2698274 DEBUG (zkCallback-8951-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:51624_solr]
   [junit4]   2> 2698274 DEBUG (zkCallback-8951-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=compute_plan, class=solr.ComputePlanAction}, {name=test, class=org.apache.solr.cloud.autoscaling.ComputePlanActionTest$AssertingTriggerAction}], enabled=true}
   [junit4]   2> 2698274 DEBUG (zkCallback-8951-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
   [junit4]   2> 2698275 DEBUG (zkCallback-8951-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:51624_solr]
   [junit4]   2> 2698275 DEBUG (zkCallback-8951-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=compute_plan, class=solr.ComputePlanAction}, {name=test, class=org.apache.solr.cloud.autoscaling.ComputePlanActionTest$AssertingTriggerAction}], enabled=true}
   [junit4]   2> 2698275 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
   [junit4]   2> 2698279 INFO  (OverseerThreadFactory-9947-thread-1) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection testNodeAdded
   [junit4]   2> 2698284 WARN  (OverseerThreadFactory-9947-thread-1) [    ] o.a.s.c.a.c.CreateCollectionCmd Specified number of replicas of 2 on collection testNodeAdded is higher than the number of Solr instances currently live or live and part of your createNodeSet(1). It's unusual to run two replica of the same slice on the same Solr-instance.
   [junit4]   2> 2698286 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 2698287 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
   [junit4]   2> 2698287 INFO  (qtp33066169-30773) [n:127.0.0.1:51624_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 2698289 DEBUG (ScheduledTrigger-9945-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 2698289 DEBUG (ScheduledTrigger-9945-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 2698291 INFO  (OverseerThreadFactory-9947-thread-1) [    ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 1729993638 sessionWrapper.createTime 1729993635313537, this.sessionWrapper.createTime 1729993635313537 
   [junit4]   2> 2698302 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "name":"testNodeAdded",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"2",
   [junit4]   2>   "collection.configName":"conf",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "nrtReplicas":"2",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"} current state version: 0
   [junit4]   2> 2698302 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.ClusterStateMutator building a new cName: testNodeAdded
   [junit4]   2> 2698303 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/testNodeAdded/state.json
   [junit4]   2> 2698405 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testNodeAdded",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:51624/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 2698405 INFO  (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testNodeAdded",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:51624/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 2698407 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{}}
   [junit4]   2> 2698407 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator New Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{"core_node3":{
   [junit4]   2>       "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>       "base_url":"http://127.0.0.1:51624/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 2698409 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testNodeAdded",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testNodeAdded_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:51624/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 2698409 INFO  (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testNodeAdded",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testNodeAdded_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:51624/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 2698410 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{"core_node3":{
   [junit4]   2>       "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>       "base_url":"http://127.0.0.1:51624/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 2698410 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator New Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>       "base_url":"http://127.0.0.1:51624/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testNodeAdded_shard1_replica_n2",
   [junit4]   2>       "base_url":"http://127.0.0.1:51624/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 2698510 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/testNodeAdded/state.json version: 0
   [junit4]   2> 2698612 INFO  (qtp33066169-30773) [n:127.0.0.1:51624_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=testNodeAdded_shard1_replica_n2&action=CREATE&numShards=1&collection=testNodeAdded&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 2698613 INFO  (qtp33066169-30773) [n:127.0.0.1:51624_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2698613 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=testNodeAdded_shard1_replica_n1&action=CREATE&numShards=1&collection=testNodeAdded&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 2698621 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>   "core_node_name":"core_node3",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51624/solr",
   [junit4]   2>   "node_name":"127.0.0.1:51624_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testNodeAdded",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 2698621 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>   "core_node_name":"core_node3",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51624/solr",
   [junit4]   2>   "node_name":"127.0.0.1:51624_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testNodeAdded",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2698621 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node3:{"core":"testNodeAdded_shard1_replica_n1","base_url":"http://127.0.0.1:51624/solr","node_name":"127.0.0.1:51624_solr","state":"down","type":"NRT"}
   [junit4]   2> 2698621 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(testNodeAdded//collections/testNodeAdded/state.json/1)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"2",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"2",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>           "base_url":"http://127.0.0.1:51624/solr",
   [junit4]   2>           "node_name":"127.0.0.1:51624_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node4":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n2",
   [junit4]   2>           "base_url":"http://127.0.0.1:51624/solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 2698629 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "core":"testNodeAdded_shard1_replica_n2",
   [junit4]   2>   "core_node_name":"core_node4",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51624/solr",
   [junit4]   2>   "node_name":"127.0.0.1:51624_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testNodeAdded",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 2698629 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"testNodeAdded_shard1_replica_n2",
   [junit4]   2>   "core_node_name":"core_node4",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51624/solr",
   [junit4]   2>   "node_name":"127.0.0.1:51624_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testNodeAdded",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2698629 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node4:{"core":"testNodeAdded_shard1_replica_n2","base_url":"http://127.0.0.1:51624/solr","node_name":"127.0.0.1:51624_solr","state":"down","type":"NRT"}
   [junit4]   2> 2698629 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(testNodeAdded//collections/testNodeAdded/state.json/1)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"2",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"2",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>           "base_url":"http://127.0.0.1:51624/solr",
   [junit4]   2>           "node_name":"127.0.0.1:51624_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node4":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n2",
   [junit4]   2>           "base_url":"http://127.0.0.1:51624/solr",
   [junit4]   2>           "node_name":"127.0.0.1:51624_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 2698730 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/testNodeAdded/state.json version: 1
   [junit4]   2> 2698731 INFO  (zkCallback-8944-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2698731 INFO  (zkCallback-8944-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2699296 DEBUG (ScheduledTrigger-9945-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 2699296 DEBUG (ScheduledTrigger-9945-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 2699664 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2699670 INFO  (qtp33066169-30773) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2699685 INFO  (qtp33066169-30773) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.s.IndexSchema [testNodeAdded_shard1_replica_n2] Schema name=minimal
   [junit4]   2> 2699687 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.s.IndexSchema [testNodeAdded_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 2699688 INFO  (qtp33066169-30773) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2699688 INFO  (qtp33066169-30773) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'testNodeAdded_shard1_replica_n2' using configuration from collection testNodeAdded, trusted=true
   [junit4]   2> 2699754 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2699754 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testNodeAdded_shard1_replica_n1' using configuration from collection testNodeAdded, trusted=true
   [junit4]   2> 2699796 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51624.solr.core.testNodeAdded.shard1.replica_n1' (registry 'solr.core.testNodeAdded.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ce44515
   [junit4]   2> 2699796 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2699796 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.SolrCore [[testNodeAdded_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.ComputePlanActionTest_64DEE8598928EE50-001/tempDir-001/node1/testNodeAdded_shard1_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.ComputePlanActionTest_64DEE8598928EE50-001/tempDir-001/node1/./testNodeAdded_shard1_replica_n1/data/]
   [junit4]   2> 2699800 INFO  (qtp33066169-30773) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51624.solr.core.testNodeAdded.shard1.replica_n2' (registry 'solr.core.testNodeAdded.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ce44515
   [junit4]   2> 2699800 INFO  (qtp33066169-30773) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2699800 INFO  (qtp33066169-30773) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.SolrCore [[testNodeAdded_shard1_replica_n2] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.ComputePlanActionTest_64DEE8598928EE50-001/tempDir-001/node1/testNodeAdded_shard1_replica_n2], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.ComputePlanActionTest_64DEE8598928EE50-001/tempDir-001/node1/./testNodeAdded_shard1_replica_n2/data/]
   [junit4]   2> 2700007 INFO  (qtp33066169-30773) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2700007 INFO  (qtp33066169-30773) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2700009 INFO  (qtp33066169-30773) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2700009 INFO  (qtp33066169-30773) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2700010 INFO  (qtp33066169-30773) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@14fb37f2[testNodeAdded_shard1_replica_n2] main]
   [junit4]   2> 2700011 INFO  (qtp33066169-30773) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2700012 INFO  (qtp33066169-30773) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2700015 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2700015 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2700016 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2700016 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2700017 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@40848700[testNodeAdded_shard1_replica_n1] main]
   [junit4]   2> 2700019 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2700019 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2700020 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2700020 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1596721833424453632
   [junit4]   2> 2700025 INFO  (qtp33066169-30773) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2700025 INFO  (qtp33066169-30773) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1596721833429696512
   [junit4]   2> 2700027 INFO  (searcherExecutor-9950-thread-1-processing-n:127.0.0.1:51624_solr x:testNodeAdded_shard1_replica_n2 c:testNodeAdded s:shard1 r:core_node4) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.SolrCore [testNodeAdded_shard1_replica_n2] Registered new searcher Searcher@14fb37f2[testNodeAdded_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2700050 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testNodeAdded/terms/shard1 to Terms{values={core_node3=0}, version=0}
   [junit4]   2> 2700055 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
   [junit4]   2> 2700056 INFO  (searcherExecutor-9951-thread-1-processing-n:127.0.0.1:51624_solr x:testNodeAdded_shard1_replica_n1 c:testNodeAdded s:shard1 r:core_node3) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.SolrCore [testNodeAdded_shard1_replica_n1] Registered new searcher Searcher@40848700[testNodeAdded_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2700057 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testNodeAdded"} current state version: 0
   [junit4]   2> 2700061 INFO  (qtp33066169-30773) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testNodeAdded/terms/shard1 to Terms{values={core_node3=0, core_node4=0}, version=1}
   [junit4]   2> 2700173 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/testNodeAdded/state.json version: 2
   [junit4]   2> 2700189 INFO  (zkCallback-8944-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2700192 INFO  (zkCallback-8944-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2700297 DEBUG (ScheduledTrigger-9945-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 2700297 DEBUG (ScheduledTrigger-9945-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 2700560 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2700560 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2700560 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:51624/solr/testNodeAdded_shard1_replica_n1/
   [junit4]   2> 2700560 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=testNodeAdded_shard1_replica_n1 url=http://127.0.0.1:51624/solr START replicas=[http://127.0.0.1:51624/solr/testNodeAdded_shard1_replica_n2/] nUpdates=100
   [junit4]   2> 2700562 INFO  (qtp33066169-30774) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.S.Request [testNodeAdded_shard1_replica_n2]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2700861 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=testNodeAdded_shard1_replica_n1 url=http://127.0.0.1:51624/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 2700861 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 2700861 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 2700861 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2700869 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:51624/solr/testNodeAdded_shard1_replica_n1/ shard1
   [junit4]   2> 2700870 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testNodeAdded",
   [junit4]   2>   "base_url":"http://127.0.0.1:51624/solr",
   [junit4]   2>   "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>   "state":"active"} current state version: 0
   [junit4]   2> 2700977 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/testNodeAdded/state.json version: 3
   [junit4]   2> 2700977 INFO  (zkCallback-8944-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2700977 INFO  (zkCallback-8944-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2701026 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2701029 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>   "core_node_name":"core_node3",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51624/solr",
   [junit4]   2>   "node_name":"127.0.0.1:51624_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testNodeAdded",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 2701029 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>   "core_node_name":"core_node3",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51624/solr",
   [junit4]   2>   "node_name":"127.0.0.1:51624_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testNodeAdded",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2701029 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node3:{"core":"testNodeAdded_shard1_replica_n1","base_url":"http://127.0.0.1:51624/solr","node_name":"127.0.0.1:51624_solr","state":"active","type":"NRT","leader":"true"}
   [junit4]   2> 2701029 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(testNodeAdded//collections/testNodeAdded/state.json/4)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"2",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"2",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>           "base_url":"http://127.0.0.1:51624/solr",
   [junit4]   2>           "node_name":"127.0.0.1:51624_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "leader":"true"},
   [junit4]   2>         "core_node4":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n2",
   [junit4]   2>           "base_url":"http://127.0.0.1:51624/solr",
   [junit4]   2>           "node_name":"127.0.0.1:51624_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 2701029 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=testNodeAdded_shard1_replica_n1&action=CREATE&numShards=1&collection=testNodeAdded&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2416
   [junit4]   2> 2701076 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "core":"testNodeAdded_shard1_replica_n2",
   [junit4]   2>   "core_node_name":"core_node4",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51624/solr",
   [junit4]   2>   "node_name":"127.0.0.1:51624_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testNodeAdded",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 2701076 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"testNodeAdded_shard1_replica_n2",
   [junit4]   2>   "core_node_name":"core_node4",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:51624/solr",
   [junit4]   2>   "node_name":"127.0.0.1:51624_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testNodeAdded",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2701076 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node4:{"core":"testNodeAdded_shard1_replica_n2","base_url":"http://127.0.0.1:51624/solr","node_name":"127.0.0.1:51624_solr","state":"active","type":"NRT"}
   [junit4]   2> 2701076 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(testNodeAdded//collections/testNodeAdded/state.json/4)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"2",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"2",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>           "base_url":"http://127.0.0.1:51624/solr",
   [junit4]   2>           "node_name":"127.0.0.1:51624_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "leader":"true"},
   [junit4]   2>         "core_node4":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n2",
   [junit4]   2>           "base_url":"http://127.0.0.1:51624/solr",
   [junit4]   2>           "node_name":"127.0.0.1:51624_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 2701077 INFO  (qtp33066169-30773) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=testNodeAdded_shard1_replica_n2&action=CREATE&numShards=1&collection=testNodeAdded&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2465
   [junit4]   2> 2701082 INFO  (qtp33066169-30776) [n:127.0.0.1:51624_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 2701179 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/testNodeAdded/state.json version: 4
   [junit4]   2> 2701180 INFO  (zkCallback-8944-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2701181 INFO  (zkCallback-8944-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2701299 DEBUG (ScheduledTrigger-9945-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 2701299 DEBUG (ScheduledTrigger-9945-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 2702084 INFO  (qtp33066169-30776) [n:127.0.0.1:51624_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=conf&name=testNodeAdded&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=3811
   [junit4]   2> 2702091 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.testNodeAdded.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.testNodeAdded.shard1.replica_n2:INDEX.sizeInBytes} status=0 QTime=0
   [junit4]   2> 2702093 INFO  (qtp33066169-30774) [n:127.0.0.1:51624_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 2702098 DEBUG (qtp33066169-30778) [n:127.0.0.1:51624_solr    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 2702099 INFO  (qtp33066169-30778) [n:127.0.0.1:51624_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=12
   [junit4]   2> 2702099 DEBUG (zkCallback-8951-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
   [junit4]   2> 2702100 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 2702100 DEBUG (zkCallback-8951-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:51624_solr]
   [junit4]   2> 2702100 DEBUG (zkCallback-8951-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=compute_plan, class=solr.ComputePlanAction}, {name=test, class=org.apache.solr.cloud.autoscaling.ComputePlanActionTest$AssertingTriggerAction}], enabled=true}
   [junit4]   2> 2702100 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
   [junit4]   2> 2702101 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 2702101 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 2702101 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 2702101 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 2702101 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
   [junit4]   2> 2702101 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2e970eff{/solr,null,AVAILABLE}
   [junit4]   2> 2702102 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@6bd5252c{HTTP/1.1,[http/1.1]}{127.0.0.1:33733}
   [junit4]   2> 2702102 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [    ] o.e.j.s.Server Started @2703847ms
   [junit4]   2> 2702102 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33733}
   [junit4]   2> 2702102 ERROR (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2702105 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 2702105 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 2702105 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2702105 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2702105 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-04-03T10:50:05.067Z
   [junit4]   2> 2702107 INFO  (zkConnectionManagerCallback-8963-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2702108 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2702116 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:64060/solr
   [junit4]   2> 2702118 INFO  (zkConnectionManagerCallback-8967-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2702121 INFO  (zkConnectionManagerCallback-8969-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2702162 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [n:127.0.0.1:33733_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2702166 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [n:127.0.0.1:33733_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2702170 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [n:127.0.0.1:33733_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2702170 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [n:127.0.0.1:33733_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33733_solr
   [junit4]   2> 2702171 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:33733_solr"} current state version: 0
   [junit4]   2> 2702171 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:33733_solr
   [junit4]   2> 2702172 INFO  (zkCallback-8951-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2702172 INFO  (zkCallback-8944-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2702173 INFO  (zkCallback-8958-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2702173 INFO  (zkCallback-8958-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2702173 INFO  (zkCallback-8968-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2702287 INFO  (OverseerCollectionConfigSetProcessor-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2702300 DEBUG (ScheduledTrigger-9945-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 2702300 DEBUG (ScheduledTrigger-9945-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 2702300 DEBUG (ScheduledTrigger-9945-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:33733_solr at time 1729997647716518
   [junit4]   2> 2702882 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [n:127.0.0.1:33733_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33733.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ce44515
   [junit4]   2> 2702893 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [n:127.0.0.1:33733_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33733.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ce44515
   [junit4]   2> 2702894 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [n:127.0.0.1:33733_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33733.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4ce44515
   [junit4]   2> 2702895 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [n:127.0.0.1:33733_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.ComputePlanActionTest_64DEE8598928EE50-001/tempDir-001/node2/.
   [junit4]   2> 2702901 INFO  (zkConnectionManagerCallback-8976-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2702911 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [n:127.0.0.1:33733_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2702913 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [n:127.0.0.1:33733_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:64060/solr ready
   [junit4]   2> 2703302 DEBUG (ScheduledTrigger-9945-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 2703302 DEBUG (ScheduledTrigger-9945-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 2703302 DEBUG (ScheduledTrigger-9945-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:33733_solr] added at times [1729997647716518], now=1729998650231790
   [junit4]   2> 2703302 DEBUG (ScheduledTrigger-9945-thread-4) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"6256c774120a6Te6t08jwas7ry8o42sn93757ic",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":1729997647716518,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[1729997647716518],
   [junit4]   2>     "nodeNames":["127.0.0.1:33733_solr"]}}
   [junit4]   2> 2703305 DEBUG (ScheduledTrigger-9945-thread-4) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_trigger]
   [junit4]   2> 2703306 INFO  (ScheduledTrigger-9945-thread-4) [    ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
   [junit4]   2> 2703307 DEBUG (ScheduledTrigger-9945-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:33733_solr
   [junit4]   2> 2703307 DEBUG (AutoscalingActionExecutor-9946-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"6256c774120a6Te6t08jwas7ry8o42sn93757ic",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":1729997647716518,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[1729997647716518],
   [junit4]   2>     "_enqueue_time_":1729998653009517,
   [junit4]   2>     "nodeNames":["127.0.0.1:33733_solr"]}}
   [junit4]   2> 2703307 DEBUG (AutoscalingActionExecutor-9946-thread-1) [    ] o.a.s.c.a.ComputePlanAction -- processing event: {
   [junit4]   2>   "id":"6256c774120a6Te6t08jwas7ry8o42sn93757ic",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":1729997647716518,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[1729997647716518],
   [junit4]   2>     "_enqueue_time_":1729998653009517,
   [junit4]   2>     "nodeNames":["127.0.0.1:33733_solr"]}} with context properties: {BEFORE_ACTION=[compute_plan]}
   [junit4]   2> 2703312 INFO  (qtp580676573-30832) [n:127.0.0.1:33733_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 2703322 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.testNodeAdded.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.testNodeAdded.shard1.replica_n2:INDEX.sizeInBytes} status=0 QTime=0
   [junit4]   2> 2703323 INFO  (qtp33066169-30778) [n:127.0.0.1:51624_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 2703325 INFO  (AutoscalingActionExecutor-9946-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeAdded&targetNode=127.0.0.1:33733_solr&inPlaceMove=true&replica=core_node3
   [junit4]   2> 2703325 INFO  (AutoscalingActionExecutor-9946-thread-1) [    ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 1729998673 sessionWrapper.createTime 1729998671457444, this.sessionWrapper.createTime 1729998671457444 
   [junit4]   2> 2703327 DEBUG (AutoscalingActionExecutor-9946-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 5000ms
   [junit4]   2> 2703327 DEBUG (AutoscalingActionExecutor-9946-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 19 ms for event id=6256c774120a6Te6t08jwas7ry8o42sn93757ic
   [junit4]   2> 2703327 DEBUG (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [    ] o.a.s.c.a.ComputePlanActionTest -------------_ FINAL STATE --------------
   [junit4]   2> 2703329 INFO  (qtp580676573-30835) [n:127.0.0.1:33733_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 2703329 DEBUG (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [    ] o.a.s.c.a.ComputePlanActionTest * Node values: 127.0.0.1:33733_solr
   [junit4]   2> {
   [junit4]   2>   "node":"127.0.0.1:33733_solr",
   [junit4]   2>   "cores":0,
   [junit4]   2>   "port":"33733",
   [junit4]   2>   "host":"127.0.0.1",
   [junit4]   2>   "ip_1":"1",
   [junit4]   2>   "ip_3":"0",
   [junit4]   2>   "ip_2":"0",
   [junit4]   2>   "ip_4":"127",
   [junit4]   2>   "freedisk":12.675395488739014}
   [junit4]   2> 2703331 INFO  (qtp33066169-30779) [n:127.0.0.1:51624_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 2703331 DEBUG (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [    ] o.a.s.c.a.ComputePlanActionTest * Node values: 127.0.0.1:51624_solr
   [junit4]   2> {
   [junit4]   2>   "node":"127.0.0.1:51624_solr",
   [junit4]   2>   "cores":2,
   [junit4]   2>   "port":"51624",
   [junit4]   2>   "host":"127.0.0.1",
   [junit4]   2>   "ip_1":"1",
   [junit4]   2>   "ip_3":"0",
   [junit4]   2>   "ip_2":"0",
   [junit4]   2>   "ip_4":"127",
   [junit4]   2>   "freedisk":12.675395488739014}
   [junit4]   2> 2703331 DEBUG (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [    ] o.a.s.c.a.ComputePlanActionTest * Live nodes: [127.0.0.1:33733_solr, 127.0.0.1:51624_solr]
   [junit4]   2> 2703333 DEBUG (TEST-ComputePlanActionTest.testNodeAdded-seed#[64DEE8598928EE50]) [    ] o.a.s.c.a.ComputePlanActionTest * Collection testNodeAdded state: DocCollection(testNodeAdded//collections/testNodeAdded/state.json/5)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"2",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>           "base_url":"http://127.0.0.1:51624/solr",
   [junit4]   2>           "node_name":"127.0.0.1:51624_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "leader":"true"},
   [junit4]   2>         "core_node4":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n2",
   [junit4]   2>           "base_url":"http://127.0.0.1:51624/solr",
   [junit4]   2>           "node_name":"127.0.0.1:51624_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT"}}}},
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"2",
   [junit4]   2>   "tlogReplicas":"0"}
   [junit4]   2> 2703766 DEBUG (zkCallback-8951-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
   [junit4]   2> 2703766 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
   [junit4]   2> 2703770 INFO  (TEST-ComputePlanActionTest.testNodeLost-seed#[64DEE8598928EE50]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@56ef9d3e{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 2703771 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 2703771 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 2703771 DEBUG (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
   [junit4]   2> 2703779 INFO  (TEST-ComputePlanActionTest.testNodeLost-seed#[64DEE8598928EE50]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=942395587
   [junit4]   2> 2703806 INFO  (TEST-ComputePlanActionTest.testNodeLost-seed#[64DEE8598928EE50]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 2703806 INFO  (TEST-ComputePlanActionTest.testNodeLost-seed#[64DEE8598928EE50]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6d142fff: rootName = solr_51624, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@63a1e75
   [junit4]   2> 2703823 INFO  (TEST-ComputePlanActionTest.testNodeLost-seed#[64DEE8598928EE50]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 2703823 INFO  (TEST-ComputePlanActionTest.testNodeLost-seed#[64DEE8598928EE50]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6d0e7eef: rootName = solr_51624, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@51e67ee3
   [junit4]   2> 2703827 INFO  (TEST-ComputePlanActionTest.testNodeLost-seed#[64DEE8598928EE50]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 2703827 INFO  (TEST-ComputePlanActionTest.testNodeLost-seed#[64DEE8598928EE50]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6b7ab89b: rootName = solr_51624, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@217af968
   [junit4]   2> 2703830 INFO  (TEST-ComputePlanActionTest.testNodeLost-seed#[64DEE8598928EE50]) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:51624_solr
   [junit4]   2> 2703831 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:51624_solr"} current state version: 0
   [junit4]   2> 2703831 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:51624_solr
   [junit4]   2> 2703831 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.NodeMutator Update replica state for core_node3:{"core":"testNodeAdded_shard1_replica_n1","base_url":"http://127.0.0.1:51624/solr","node_name":"127.0.0.1:51624_solr","state":"active","type":"NRT","leader":"true"} to down
   [junit4]   2> 2703831 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.NodeMutator Update replica state for core_node4:{"core":"testNodeAdded_shard1_replica_n2","base_url":"http://127.0.0.1:51624/solr","node_name":"127.0.0.1:51624_solr","state":"active","type":"NRT"} to down
   [junit4]   2> 2703832 INFO  (zkCallback-8968-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 2703832 INFO  (zkCallback-8958-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 2703832 INFO  (zkCallback-8958-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 2703832 INFO  (TEST-ComputePlanActionTest.testNodeLost-seed#[64DEE8598928EE50]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 2703833 DEBUG (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/testNodeAdded/state.json version: 5
   [junit4]   2> 2703834 INFO  (zkCallback-8944-thread-3) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2703834 INFO  (zkCallback-8951-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 2703834 INFO  (zkCallback-8944-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2703835 INFO  (coreCloseExecutor-9966-thread-2) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.SolrCore [testNodeAdded_shard1_replica_n2]  CLOSING SolrCore org.apache.solr.core.SolrCore@673381ed
   [junit4]   2> 2703835 INFO  (coreCloseExecutor-9966-thread-1) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.SolrCore [testNodeAdded_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@1c58aef2
   [junit4]   2> 2703835 INFO  (coreCloseExecutor-9966-thread-2) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testNodeAdded.shard1.replica_n2, tag=673381ed
   [junit4]   2> 2703835 INFO  (coreCloseExecutor-9966-thread-2) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7cdba43d: rootName = solr_51624, domain = solr.core.testNodeAdded.shard1.replica_n2, service url = null, agent id = null] for registry solr.core.testNodeAdded.shard1.replica_n2 / com.codahale.metrics.MetricRegistry@61437144
   [junit4]   2> 2703836 INFO  (zkCallback-8944-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 2703838 INFO  (zkCallback-8975-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 2703843 INFO  (coreCloseExecutor-9966-thread-2) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testNodeAdded.shard1.leader, tag=673381ed
   [junit4]   2> 2703843 INFO  (coreCloseExecutor-9966-thread-1) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testNodeAdded.shard1.replica_n1, tag=1c58aef2
   [junit4]   2> 2703843 INFO  (coreCloseExecutor-9966-thread-1) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@68495f76: rootName = solr_51624, domain = solr.core.testNodeAdded.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.testNodeAdded.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@32d3b23f
   [junit4]   2> 2703851 INFO  (coreCloseExecutor-9966-thread-1) [n:127.0.0.1:51624_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testNodeAdded.shard1.leader, tag=1c58aef2
   [junit4]   2> 2703853 INFO  (TEST-ComputePlanActionTest.testNodeLost-seed#[64DEE8598928EE50]) [    ] o.a.s.c.Overseer Overseer (id=72170970746585091-127.0.0.1:51624_solr-n_0000000000) closing
   [junit4]   2> 2703853 INFO  (OverseerStateUpdate-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:51624_solr
   [junit4]   2> 2703854 WARN  (OverseerAutoScalingTriggerThread-72170970746585091-127.0.0.1:51624_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 2703854 DEBUG (TEST-ComputePlanActionTest.testNodeLost-seed#[64DEE8598928EE50]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 2703855 DEBUG (TEST-ComputePlanActionTest.testNodeLost-seed#[64DEE8598928EE50]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 2703855 DEBUG (TEST-ComputePlanActionTest.testNodeLost-seed#[64DEE8598928EE50]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 2703855 DEBUG (TEST-ComputePlanActionTest.testNodeLost-seed#[64DEE8598928EE50]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 2703855 DEBUG (TEST-ComputePlanActionTest.testNodeLost-seed#[64DEE8598928EE50]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 2703855 DEBUG (TEST-ComputePlanActionTest.testNodeLost-seed#[64DEE8598928EE50]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 2703870 INFO  (TEST-ComputePlanActionTest.testNodeLost-seed#[64DEE8598928EE50]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3aaf8a51{/solr,null,UNAVAILABLE}
   [junit4]   2> 2703870 INFO  (zkCallback-8968-thread-1) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33733_solr
   [junit4]   2> 2703873 INFO  (TEST-ComputePlanActionTest.testNodeLost-seed#[64DEE8598928EE50]) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 2703875 INFO  (zkCallback-8968-thread-1) [    ] o.a.s.c.Overseer Overseer (id=72170970746585097-127.0.0.1:33733_solr-n_0000000001) starting
   [junit4]   2> 2703877 INFO  (TEST-ComputePlanActionTest.testNodeLost-seed#[64DEE8598928EE50]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2703887 INFO  (qtp580676573-30835) [n:127.0.0.1:33733_solr    ] o.a.s.h.a.CollectionsHandler Invoked Col

[...truncated too long message...]

election/72170970746585119-127.0.0.1:46039_solr-n_0000000006
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	65	/solr/live_nodes
   [junit4]   2> 	56	/solr/collections
   [junit4]   2> 	6	/solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 	5	/solr/autoscaling/events/.scheduled_maintenance
   [junit4]   2> 	5	/solr/overseer/queue
   [junit4]   2> 	5	/solr/overseer/collection-queue-work
   [junit4]   2> 	5	/solr/overseer/queue-work
   [junit4]   2> 	2	/solr/autoscaling/events/node_lost_trigger
   [junit4]   2> 
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.ComputePlanActionTest_64DEE8598928EE50-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1375, maxMBSortInHeap=6.6184234518324585, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@356880a9), locale=sq, timezone=Europe/Ulyanovsk
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_162 (64-bit)/cpus=3,threads=1,free=93061608,total=497352704
   [junit4]   2> NOTE: All tests run in this JVM: [HdfsTlogReplayBufferedWhileIndexingTest, TestHdfsUpdateLog, TestEmbeddedSolrServerSchemaAPI, TestExecutePlanAction, SpellCheckComponentTest, UUIDFieldTest, TestBadConfig, TermVectorComponentTest, TestAddFieldRealTimeGet, TestConfigSetsAPIZkFailure, TolerantUpdateProcessorTest, TestLargeCluster, EchoParamsTest, QueryParsingTest, MoveReplicaTest, TestEmbeddedSolrServerAdminHandler, TestNestedDocsSort, TestJsonFacetsWithNestedObjects, TestCoreDiscovery, TestWordDelimiterFilterFactory, TestNodeLostTrigger, TestSearcherReuse, LeaderVoteWaitTimeoutTest, LargeFieldTest, DistanceUnitsTest, HighlighterConfigTest, AnalyticsMergeStrategyTest, V2ApiIntegrationTest, TestFilteredDocIdSet, TestMacros, TestDocBasedVersionConstraints, TestSkipOverseerOperations, DOMUtilTest, TestSolrCoreSnapshots, TestCrossCoreJoin, CurrencyRangeFacetCloudTest, TestCloudSchemaless, VersionInfoTest, CollectionTooManyReplicasTest, TestRawResponseWriter, AssignBackwardCompatibilityTest, PeerSyncTest, TestExclusionRuleCollectionAccess, HdfsLockFactoryTest, TestReversedWildcardFilterFactory, PKIAuthenticationIntegrationTest, DocExpirationUpdateProcessorFactoryTest, TestDistributedMap, BigEndianAscendingWordSerializerTest, TestBlobHandler, TestHdfsCloudBackupRestore, HttpTriggerListenerTest, TestZkAclsWithHadoopAuth, HdfsSyncSliceTest, TestComponentsName, TestExtendedDismaxParser, TestDistributedMissingSort, TestStandardQParsers, OverseerTest, ScriptEngineTest, SearchRateTriggerIntegrationTest, TestSmileRequest, BadComponentTest, XsltUpdateRequestHandlerTest, PeerSyncReplicationTest, TestQuerySenderListener, TestConfigSetsAPI, TestHashQParserPlugin, UpdateParamsTest, TestComplexPhraseLeadingWildcard, HttpPartitionTest, NotRequiredUniqueKeyTest, StatsReloadRaceTest, HdfsBasicDistributedZk2Test, ExitableDirectoryReaderTest, RestoreTriggerStateTest, PreAnalyzedFieldTest, MigrateRouteKeyTest, TriggerSetPropertiesIntegrationTest, TestRandomFlRTGCloud, TestTolerantUpdateProcessorCloud, HdfsRecoveryZkTest, TestSimDistributedQueue, TestFieldCache, RestartWhileUpdatingTest, TestFreeTextSuggestions, TestPayloadCheckQParserPlugin, CustomCollectionTest, OverriddenZkACLAndCredentialsProvidersTest, HdfsWriteToMultipleCollectionsTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, TestPullReplica, TestDocumentBuilder, SortByFunctionTest, SolrMetricManagerTest, TestLegacyFieldCache, TestConfigOverlay, TestStressLiveNodes, TestManagedResource, TestFuzzyAnalyzedSuggestions, BadCopyFieldTest, ConnectionReuseTest, TestTrieFacet, ShowFileRequestHandlerTest, CoreAdminOperationTest, SpellCheckCollatorWithCollapseTest, TestAuthenticationFramework, SuggestComponentContextFilterQueryTest, SSLMigrationTest, TestDocSet, HLLSerializationTest, DistributedFacetExistsSmallTest, TestManagedSynonymFilterFactory, NoCacheHeaderTest, TestConfig, ClusterStateUpdateTest, ExternalFileFieldSortTest, CSVRequestHandlerTest, FileBasedSpellCheckerTest, ChangedSchemaMergeTest, TestLRUCache, TestFastWriter, ReplaceNodeTest, DocValuesTest, AtomicUpdateProcessorFactoryTest, TestTestInjection, RankQueryTest, CoreSorterTest, URLClassifyProcessorTest, DistanceFunctionTest, TestRandomFaceting, ConnectionManagerTest, TestImplicitCoreProperties, SolrCloudExampleTest, ZkStateWriterTest, TestSlowCompositeReaderWrapper, GraphQueryTest, TokenizerChainTest, DateRangeFieldTest, CdcrReplicationHandlerTest, TestChildDocTransformer, MoveReplicaHDFSFailoverTest, TestCustomSort, AutoAddReplicasIntegrationTest, TestSolrDeletionPolicy2, TestSolr4Spatial, HighlighterMaxOffsetTest, AddReplicaTest, TestSolrIndexConfig, TestValueSourceCache, MetricsHandlerTest, OverseerStatusTest, TestReloadAndDeleteDocs, TestSolrCloudWithKerberosAlt, HdfsNNFailoverTest, ExplicitHLLTest, BlobRepositoryCloudTest, BJQParserTest, TestRestoreCore, TestSolrConfigHandlerConcurrent, DistribDocExpirationUpdateProcessorTest, BufferStoreTest, CloudExitableDirectoryReaderTest, BasicDistributedZk2Test, TestUniqueKeyFieldResource, IndexSchemaTest, SpellPossibilityIteratorTest, TestRestManager, BlockJoinFacetRandomTest, InfixSuggestersTest, TestPHPSerializedResponseWriter, TestJsonFacetRefinement, TestCollectionAPI, TestCodecSupport, RegexBytesRefFilterTest, CdcrBootstrapTest, BasicAuthIntegrationTest, TestOrdValues, LIRRollingUpdatesTest, BlockCacheTest, TestSort, SimplePostToolTest, TestSQLHandlerNonCloud, ReturnFieldsTest, TestDynamicFieldResource, QueryResultKeyTest, ResponseLogComponentTest, SolrCmdDistributorTest, RegexBoostProcessorTest, TestCSVLoader, TestSchemaResource, TestScoreJoinQPScore, TestFieldCacheVsDocValues, ChaosMonkeySafeLeaderTest, SyncSliceTest, OpenCloseCoreStressTest, LeaderElectionIntegrationTest, ShardRoutingTest, BasicZkTest, FullSolrCloudDistribCmdsTest, LeaderElectionTest, ZkSolrClientTest, TestRandomDVFaceting, TestDistributedSearch, TestDistributedGrouping, TestRecovery, TestHashPartitioner, DistributedSpellCheckComponentTest, ZkControllerTest, DistributedQueryElevationComponentTest, AnalysisAfterCoreReloadTest, SpellCheckCollatorTest, CoreAdminHandlerTest, SolrRequestParserTest, SuggesterTest, TestStressLucene, TestTrie, SpatialFilterTest, SuggesterWFSTTest, PolyFieldTest, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, SolrCoreCheckLockOnStartupTest, TestPseudoReturnFields, TestUpdate, TestAtomicUpdateErrorCases, QueryEqualityTest, DirectUpdateHandlerOptimizeTest, DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, TestRemoteStreaming, DocValuesMultiTest, TestSolrDeletionPolicy1, SolrInfoBeanTest, DebugComponentTest, CacheHeaderTest, IndexBasedSpellCheckerTest, TestSurroundQueryParser, DisMaxRequestHandlerTest, TestQueryUtils, TestWriterPerf, DocumentAnalysisRequestHandlerTest, TestQueryTypes, PrimitiveFieldTypeTest, TermsComponentTest, RequestHandlersTest, TestIndexingPerformance, MoreLikeThisHandlerTest, RequiredFieldsTest, FieldAnalysisRequestHandlerTest, TestSolrQueryParser, LoggingHandlerTest, TestCollationField, UpdateRequestProcessorFactoryTest, MBeansHandlerTest, TestAnalyzedSuggestions, UniqFieldsUpdateProcessorFactoryTest, JsonLoaderTest, BinaryUpdateRequestHandlerTest, SearchHandlerTest, TestLFUCache, SOLR749Test, AlternateDirectoryTest, CopyFieldTest, TestStressRecovery, TestSearchPerf, MinimalSchemaTest, TestElisionMultitermQuery, TestSolrCoreProperties, DirectSolrConnectionTest, SynonymTokenizerTest, TestDFRSimilarityFactory, TestLMDirichletSimilarityFactory, TestBM25SimilarityFactory, ResourceLoaderTest, TestFastLRUCache, ChaosMonkeyNothingIsSafeTest, AsyncCallRequestStatusResponseTest, CleanupOldIndexTest, CollectionStateFormat2Test, CollectionsAPISolrJTest, DeleteReplicaTest, ForceLeaderTest, HealthCheckHandlerTest, LIROnShardRestartTest, LeaderFailoverAfterPartitionTest, MultiThreadedOCPTest, NodeMutatorTest, OverseerModifyCollectionTest, ReplaceNodeNoTargetTest, ReplicationFactorTest, SharedFSAutoReplicaFailoverTest, SolrXmlInZkTest, TestCloudInspectUtil, TestCloudJSONFacetJoinDomain, TestCryptoKeys, TestDeleteCollectionOnDownNodes, TestLeaderElectionZkExpiry, TestLeaderInitiatedRecoveryThread, TestLockTree, TestMiniSolrCloudClusterSSL, TestRebalanceLeaders, TestSolrCloudWithSecureImpersonation, TestStressInPlaceUpdates, AssignTest, CollectionsAPIDistributedZkTest, SimpleCollectionCreateDeleteTest, TestCollectionsAPIViaSolrCloudCluster, TestReplicaProperties, AutoAddReplicasPlanActionTest, AutoScalingHandlerTest, ComputePlanActionTest]
   [junit4] Completed [605/795 (1!)] on J1 in 166.64s, 4 tests, 1 failure <<< FAILURES!

[...truncated 50293 lines...]
[repro] Jenkins log URL: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1779/consoleText

[repro] Revision: 00eab54f9d6232c68a93f10ff20e3a724ffeca14

[repro] Ant options: "-Dargs=-XX:+UseCompressedOops -XX:+UseConcMarkSweepGC"
[repro] Repro line:  ant test  -Dtestcase=ComputePlanActionTest -Dtests.method=testSelectedCollections -Dtests.seed=64DEE8598928EE50 -Dtests.slow=true -Dtests.locale=sq -Dtests.timezone=Europe/Ulyanovsk -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]    solr/core
[repro]       ComputePlanActionTest
[repro] ant compile-test

[...truncated 3318 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.ComputePlanActionTest" -Dtests.showOutput=onerror "-Dargs=-XX:+UseCompressedOops -XX:+UseConcMarkSweepGC" -Dtests.seed=64DEE8598928EE50 -Dtests.slow=true -Dtests.locale=sq -Dtests.timezone=Europe/Ulyanovsk -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[...truncated 83 lines...]
[repro] Failures:
[repro]   0/5 failed: org.apache.solr.cloud.autoscaling.ComputePlanActionTest
[repro] Exiting with code 0

[...truncated 70 lines...]

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1780 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1780/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC

21 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testTriggerThrottling

Error Message:
Both triggers should have fired by now

Stack Trace:
java.lang.AssertionError: Both triggers should have fired by now
	at __randomizedtesting.SeedInfo.seed([BD0B4D436E7EE9EE:4629E566BCD40A7C]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testTriggerThrottling(TestTriggerIntegration.java:225)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.search.TestRecovery.testBuffering

Error Message:


Stack Trace:
java.lang.NullPointerException
	at __randomizedtesting.SeedInfo.seed([BD0B4D436E7EE9EE:A0E5E368CF2748C5]:0)
	at org.apache.solr.search.TestRecovery.testBuffering(TestRecovery.java:495)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.search.TestRecovery.testDropBuffered

Error Message:
expected:<ACTIVE> but was:<BUFFERING>

Stack Trace:
java.lang.AssertionError: expected:<ACTIVE> but was:<BUFFERING>
	at __randomizedtesting.SeedInfo.seed([BD0B4D436E7EE9EE:99F76611BB309CF2]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.search.TestRecovery.testDropBuffered(TestRecovery.java:685)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.search.TestRecovery.testRecoveryMultipleLogs

Error Message:
mismatch: '6'!='3' @ response/numFound

Stack Trace:
java.lang.RuntimeException: mismatch: '6'!='3' @ response/numFound
	at __randomizedtesting.SeedInfo.seed([BD0B4D436E7EE9EE:49ED572E905AE07]:0)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:1002)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:949)
	at org.apache.solr.search.TestRecovery.testRecoveryMultipleLogs(TestRecovery.java:1448)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.search.TestRecovery.testCorruptLog

Error Message:
mismatch: '3'!='0' @ response/numFound

Stack Trace:
java.lang.RuntimeException: mismatch: '3'!='0' @ response/numFound
	at __randomizedtesting.SeedInfo.seed([BD0B4D436E7EE9EE:4FD6F9909446B832]:0)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:1002)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:949)
	at org.apache.solr.search.TestRecovery.testCorruptLog(TestRecovery.java:1368)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.search.TestRecovery.testBuffering

Error Message:


Stack Trace:
java.lang.NullPointerException
	at __randomizedtesting.SeedInfo.seed([BD0B4D436E7EE9EE:A0E5E368CF2748C5]:0)
	at org.apache.solr.search.TestRecovery.testBuffering(TestRecovery.java:495)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.search.TestRecovery.testDropBuffered

Error Message:
expected:<ACTIVE> but was:<BUFFERING>

Stack Trace:
java.lang.AssertionError: expected:<ACTIVE> but was:<BUFFERING>
	at __randomizedtesting.SeedInfo.seed([BD0B4D436E7EE9EE:99F76611BB309CF2]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.search.TestRecovery.testDropBuffered(TestRecovery.java:685)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.search.TestRecovery.testCleanShutdown

Error Message:
expected:<10> but was:<7>

Stack Trace:
java.lang.AssertionError: expected:<10> but was:<7>
	at __randomizedtesting.SeedInfo.seed([BD0B4D436E7EE9EE:2344F5840BE7B1AB]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.junit.Assert.assertEquals(Assert.java:456)
	at org.apache.solr.search.TestRecovery.testCleanShutdown(TestRecovery.java:1105)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.search.TestRecovery.testRecoveryMultipleLogs

Error Message:
mismatch: '6'!='3' @ response/numFound

Stack Trace:
java.lang.RuntimeException: mismatch: '6'!='3' @ response/numFound
	at __randomizedtesting.SeedInfo.seed([BD0B4D436E7EE9EE:49ED572E905AE07]:0)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:1002)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:949)
	at org.apache.solr.search.TestRecovery.testRecoveryMultipleLogs(TestRecovery.java:1448)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.search.TestRecovery.testCorruptLog

Error Message:
mismatch: '3'!='0' @ response/numFound

Stack Trace:
java.lang.RuntimeException: mismatch: '3'!='0' @ response/numFound
	at __randomizedtesting.SeedInfo.seed([BD0B4D436E7EE9EE:4FD6F9909446B832]:0)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:1002)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:949)
	at org.apache.solr.search.TestRecovery.testCorruptLog(TestRecovery.java:1368)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.search.TestRecovery.testBuffering

Error Message:


Stack Trace:
java.lang.NullPointerException
	at __randomizedtesting.SeedInfo.seed([BD0B4D436E7EE9EE:A0E5E368CF2748C5]:0)
	at org.apache.solr.search.TestRecovery.testBuffering(TestRecovery.java:495)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.search.TestRecovery.testBuffering

Error Message:


Stack Trace:
java.lang.NullPointerException
	at __randomizedtesting.SeedInfo.seed([BD0B4D436E7EE9EE:A0E5E368CF2748C5]:0)
	at org.apache.solr.search.TestRecovery.testBuffering(TestRecovery.java:495)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.search.TestRecovery.testDropBuffered

Error Message:
expected:<ACTIVE> but was:<BUFFERING>

Stack Trace:
java.lang.AssertionError: expected:<ACTIVE> but was:<BUFFERING>
	at __randomizedtesting.SeedInfo.seed([BD0B4D436E7EE9EE:99F76611BB309CF2]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.search.TestRecovery.testDropBuffered(TestRecovery.java:685)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.search.TestRecovery.testCleanShutdown

Error Message:
expected:<10> but was:<8>

Stack Trace:
java.lang.AssertionError: expected:<10> but was:<8>
	at __randomizedtesting.SeedInfo.seed([BD0B4D436E7EE9EE:2344F5840BE7B1AB]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.junit.Assert.assertEquals(Assert.java:456)
	at org.apache.solr.search.TestRecovery.testCleanShutdown(TestRecovery.java:1105)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.search.TestRecovery.testRecoveryMultipleLogs

Error Message:
mismatch: '6'!='3' @ response/numFound

Stack Trace:
java.lang.RuntimeException: mismatch: '6'!='3' @ response/numFound
	at __randomizedtesting.SeedInfo.seed([BD0B4D436E7EE9EE:49ED572E905AE07]:0)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:1002)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:949)
	at org.apache.solr.search.TestRecovery.testRecoveryMultipleLogs(TestRecovery.java:1448)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.search.TestRecovery.testCorruptLog

Error Message:
mismatch: '3'!='0' @ response/numFound

Stack Trace:
java.lang.RuntimeException: mismatch: '3'!='0' @ response/numFound
	at __randomizedtesting.SeedInfo.seed([BD0B4D436E7EE9EE:4FD6F9909446B832]:0)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:1002)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:949)
	at org.apache.solr.search.TestRecovery.testCorruptLog(TestRecovery.java:1368)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.search.TestRecovery.testBuffering

Error Message:


Stack Trace:
java.lang.NullPointerException
	at __randomizedtesting.SeedInfo.seed([BD0B4D436E7EE9EE:A0E5E368CF2748C5]:0)
	at org.apache.solr.search.TestRecovery.testBuffering(TestRecovery.java:495)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.search.TestRecovery.testDropBuffered

Error Message:
expected:<ACTIVE> but was:<BUFFERING>

Stack Trace:
java.lang.AssertionError: expected:<ACTIVE> but was:<BUFFERING>
	at __randomizedtesting.SeedInfo.seed([BD0B4D436E7EE9EE:99F76611BB309CF2]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.search.TestRecovery.testDropBuffered(TestRecovery.java:685)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.search.TestRecovery.testCleanShutdown

Error Message:
expected:<10> but was:<9>

Stack Trace:
java.lang.AssertionError: expected:<10> but was:<9>
	at __randomizedtesting.SeedInfo.seed([BD0B4D436E7EE9EE:2344F5840BE7B1AB]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.junit.Assert.assertEquals(Assert.java:456)
	at org.apache.solr.search.TestRecovery.testCleanShutdown(TestRecovery.java:1105)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.search.TestRecovery.testRecoveryMultipleLogs

Error Message:
mismatch: '6'!='3' @ response/numFound

Stack Trace:
java.lang.RuntimeException: mismatch: '6'!='3' @ response/numFound
	at __randomizedtesting.SeedInfo.seed([BD0B4D436E7EE9EE:49ED572E905AE07]:0)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:1002)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:949)
	at org.apache.solr.search.TestRecovery.testRecoveryMultipleLogs(TestRecovery.java:1448)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.search.TestRecovery.testCorruptLog

Error Message:
mismatch: '3'!='0' @ response/numFound

Stack Trace:
java.lang.RuntimeException: mismatch: '3'!='0' @ response/numFound
	at __randomizedtesting.SeedInfo.seed([BD0B4D436E7EE9EE:4FD6F9909446B832]:0)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:1002)
	at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:949)
	at org.apache.solr.search.TestRecovery.testCorruptLog(TestRecovery.java:1368)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 1853 lines...]
   [junit4] JVM J1: stdout was not empty, see: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/build/core/test/temp/junit4-J1-20180403_181729_3558946531573525929386.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] codec: FastDecompressionCompressingStoredFields, pf: TestBloomFilteredLucenePostings, dvf: Asserting
   [junit4] <<< JVM J1: EOF ----

[...truncated 10549 lines...]
   [junit4] Suite: org.apache.solr.search.TestRecovery
   [junit4]   2> 847963 INFO  (SUITE-TestRecovery-seed#[BD0B4D436E7EE9EE]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.search.TestRecovery_BD0B4D436E7EE9EE-001/init-core-data-001
   [junit4]   2>

[...truncated too long message...]

.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 171128 INFO  (coreLoadExecutor-749-thread-1-processing-x:collection1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 171129 INFO  (coreLoadExecutor-749-thread-1-processing-x:collection1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=272033416, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 171130 INFO  (coreLoadExecutor-749-thread-1-processing-x:collection1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4a2de4fb[collection1] main]
   [junit4]   2> 171131 WARN  (coreLoadExecutor-749-thread-1-processing-x:collection1) [    x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf; switching to use InMemory storage instead.
   [junit4]   2> 171131 INFO  (coreLoadExecutor-749-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 171131 INFO  (searcherExecutor-750-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4a2de4fb[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_21(8.0.0):c2)))}
   [junit4]   2> 171137 INFO  (TEST-TestRecovery.testVersionsOnRestart-seed#[BD0B4D436E7EE9EE]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[D1 (1596757074493046784)]} 0 3
   [junit4]   2> 171137 INFO  (TEST-TestRecovery.testVersionsOnRestart-seed#[BD0B4D436E7EE9EE]) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1596757074497241088,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 171137 INFO  (TEST-TestRecovery.testVersionsOnRestart-seed#[BD0B4D436E7EE9EE]) [    x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@7c558a48 commitCommandVersion:1596757074497241088
   [junit4]   2> 171167 INFO  (TEST-TestRecovery.testVersionsOnRestart-seed#[BD0B4D436E7EE9EE]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7a032f1e[collection1] main]
   [junit4]   2> 171167 INFO  (TEST-TestRecovery.testVersionsOnRestart-seed#[BD0B4D436E7EE9EE]) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 171168 INFO  (searcherExecutor-750-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7a032f1e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_21(8.0.0):c2/1:delGen=1) Uninverting(_22(8.0.0):c1)))}
   [junit4]   2> 171168 INFO  (TEST-TestRecovery.testVersionsOnRestart-seed#[BD0B4D436E7EE9EE]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 31
   [junit4]   2> 171169 INFO  (TEST-TestRecovery.testVersionsOnRestart-seed#[BD0B4D436E7EE9EE]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:D1&wt=xml} hits=1 status=0 QTime=0
   [junit4]   2> 171169 INFO  (TEST-TestRecovery.testVersionsOnRestart-seed#[BD0B4D436E7EE9EE]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={qt=/get&getVersions=2&wt=xml} status=0 QTime=0
   [junit4]   2> 171169 INFO  (TEST-TestRecovery.testVersionsOnRestart-seed#[BD0B4D436E7EE9EE]) [    x:collection1] o.a.s.SolrTestCaseJ4 ###Ending testVersionsOnRestart
   [junit4] OK      0.32s J1 | TestRecovery.testVersionsOnRestart
   [junit4]   2> 171170 INFO  (SUITE-TestRecovery-seed#[BD0B4D436E7EE9EE]-worker) [    x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 171170 INFO  (SUITE-TestRecovery-seed#[BD0B4D436E7EE9EE]-worker) [    x:collection1] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1361902807
   [junit4]   2> 171170 INFO  (SUITE-TestRecovery-seed#[BD0B4D436E7EE9EE]-worker) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 171170 INFO  (SUITE-TestRecovery-seed#[BD0B4D436E7EE9EE]-worker) [    x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@b2170b0: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@144628a5
   [junit4]   2> 171174 INFO  (SUITE-TestRecovery-seed#[BD0B4D436E7EE9EE]-worker) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 171174 INFO  (SUITE-TestRecovery-seed#[BD0B4D436E7EE9EE]-worker) [    x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2cd71edc: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@3280c6bf
   [junit4]   2> 171174 INFO  (SUITE-TestRecovery-seed#[BD0B4D436E7EE9EE]-worker) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 171174 INFO  (SUITE-TestRecovery-seed#[BD0B4D436E7EE9EE]-worker) [    x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7f646f21: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@5c709999
   [junit4]   2> 171175 INFO  (coreCloseExecutor-755-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@703909a1
   [junit4]   2> 171175 INFO  (coreCloseExecutor-755-thread-1-processing-x:collection1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=703909a1
   [junit4]   2> 171175 INFO  (coreCloseExecutor-755-thread-1-processing-x:collection1) [    x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@484ab370: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@60fc7742
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.search.TestRecovery_BD0B4D436E7EE9EE-002
   [junit4]   2> Apr 03, 2018 8:10:11 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {_root_=PostingsFormat(name=Asserting), id=Lucene50(blocksize=128)}, docValues:{_version_=DocValuesFormat(name=Asserting), val_i=DocValuesFormat(name=Memory), val_i_dvo=DocValuesFormat(name=Lucene70), id=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=1438, maxMBSortInHeap=6.032751407841564, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@2ef8894), locale=lv, timezone=Asia/Kabul
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_162 (64-bit)/cpus=3,threads=1,free=139713528,total=293601280
   [junit4]   2> NOTE: All tests run in this JVM: [TestRecovery, TestTriggerIntegration, TestTriggerIntegration, TestRecovery]
   [junit4] Completed [10/10 (5!)] on J1 in 70.81s, 19 tests, 2 failures, 3 errors <<< FAILURES!

[...truncated 19 lines...]
BUILD FAILED
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:1568: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:1092: There were test failures: 10 suites, 150 tests, 13 errors, 6 failures, 49 ignored (16 assumptions) [seed: BD0B4D436E7EE9EE]

Total time: 2 minutes 54 seconds

[repro] Setting last failure code to 256

[repro] Failures:
[repro]   0/5 failed: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration
[repro]   5/5 failed: org.apache.solr.search.TestRecovery
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=a6e24b20986ea8dfc99ab7ea21e4c48981319a35, workspace=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris
[WARNINGS] Computing warning deltas based on reference build #1779
Recording test results
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2