You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2018/06/13 23:40:55 UTC
[JENKINS] Lucene-Solr-master-Linux (64bit/jdk-10) - Build # 22245 -
Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/22245/
Java: 64bit/jdk-10 -XX:+UseCompressedOops -XX:+UseSerialGC
1 tests failed.
FAILED: org.apache.solr.cloud.TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader
Error Message:
Doc with id=4 not found in https://127.0.0.1:35021/solr/outOfSyncReplicasCannotBecomeLeader-false due to: Path not found: /id; rsp={doc=null}
Stack Trace:
java.lang.AssertionError: Doc with id=4 not found in https://127.0.0.1:35021/solr/outOfSyncReplicasCannotBecomeLeader-false due to: Path not found: /id; rsp={doc=null}
at __randomizedtesting.SeedInfo.seed([8FF9B23B57A9E206:F112922B94CEED3C]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.TestCloudConsistency.assertDocExists(TestCloudConsistency.java:252)
at org.apache.solr.cloud.TestCloudConsistency.assertDocsExistInAllReplicas(TestCloudConsistency.java:236)
at org.apache.solr.cloud.TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader(TestCloudConsistency.java:129)
at org.apache.solr.cloud.TestCloudConsistency.testOutOfSyncReplicasCannotBecomeLeader(TestCloudConsistency.java:92)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
Build Log:
[...truncated 14378 lines...]
[junit4] Suite: org.apache.solr.cloud.TestCloudConsistency
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudConsistency_8FF9B23B57A9E206-001/init-core-data-001
[junit4] 2> 1244654 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1244654 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0, clientAuth=0.0/0.0)
[junit4] 2> 1244655 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 1244655 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudConsistency_8FF9B23B57A9E206-001/tempDir-001
[junit4] 2> 1244655 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1244655 INFO (Thread-4530) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1244655 INFO (Thread-4530) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1244657 ERROR (Thread-4530) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1244755 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:40559
[junit4] 2> 1244757 INFO (zkConnectionManagerCallback-3970-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1244760 INFO (jetty-launcher-3967-thread-4) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 10+46
[junit4] 2> 1244760 INFO (jetty-launcher-3967-thread-1) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 10+46
[junit4] 2> 1244760 INFO (jetty-launcher-3967-thread-3) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 10+46
[junit4] 2> 1244760 INFO (jetty-launcher-3967-thread-2) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 10+46
[junit4] 2> 1244761 INFO (jetty-launcher-3967-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1244761 INFO (jetty-launcher-3967-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1244761 INFO (jetty-launcher-3967-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1244761 INFO (jetty-launcher-3967-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1244761 INFO (jetty-launcher-3967-thread-1) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 1244761 INFO (jetty-launcher-3967-thread-2) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 1244761 INFO (jetty-launcher-3967-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3c16ae9a{/solr,null,AVAILABLE}
[junit4] 2> 1244761 INFO (jetty-launcher-3967-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5ef3d631{/solr,null,AVAILABLE}
[junit4] 2> 1244761 INFO (jetty-launcher-3967-thread-4) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1244761 INFO (jetty-launcher-3967-thread-4) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1244761 INFO (jetty-launcher-3967-thread-4) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 1244762 INFO (jetty-launcher-3967-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@30d84aea{SSL,[ssl, http/1.1]}{127.0.0.1:35091}
[junit4] 2> 1244762 INFO (jetty-launcher-3967-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@25159a64{SSL,[ssl, http/1.1]}{127.0.0.1:36443}
[junit4] 2> 1244762 INFO (jetty-launcher-3967-thread-2) [ ] o.e.j.s.Server Started @1244782ms
[junit4] 2> 1244762 INFO (jetty-launcher-3967-thread-1) [ ] o.e.j.s.Server Started @1244782ms
[junit4] 2> 1244762 INFO (jetty-launcher-3967-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1244762 INFO (jetty-launcher-3967-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1244762 INFO (jetty-launcher-3967-thread-3) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 1244762 INFO (jetty-launcher-3967-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36443}
[junit4] 2> 1244762 INFO (jetty-launcher-3967-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35091}
[junit4] 2> 1244762 INFO (jetty-launcher-3967-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1d6e67f5{/solr,null,AVAILABLE}
[junit4] 2> 1244763 INFO (jetty-launcher-3967-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@62913173{/solr,null,AVAILABLE}
[junit4] 2> 1244763 ERROR (jetty-launcher-3967-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1244763 ERROR (jetty-launcher-3967-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1244763 INFO (jetty-launcher-3967-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1244763 INFO (jetty-launcher-3967-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1244763 INFO (jetty-launcher-3967-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1244763 INFO (jetty-launcher-3967-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1244763 INFO (jetty-launcher-3967-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1244763 INFO (jetty-launcher-3967-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1244763 INFO (jetty-launcher-3967-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1244763 INFO (jetty-launcher-3967-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-13T23:15:18.640268Z
[junit4] 2> 1244763 INFO (jetty-launcher-3967-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1244763 INFO (jetty-launcher-3967-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-13T23:15:18.640311Z
[junit4] 2> 1244763 INFO (jetty-launcher-3967-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@641a6f21{SSL,[ssl, http/1.1]}{127.0.0.1:41719}
[junit4] 2> 1244763 INFO (jetty-launcher-3967-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@109f300b{SSL,[ssl, http/1.1]}{127.0.0.1:42663}
[junit4] 2> 1244763 INFO (jetty-launcher-3967-thread-3) [ ] o.e.j.s.Server Started @1244783ms
[junit4] 2> 1244763 INFO (jetty-launcher-3967-thread-4) [ ] o.e.j.s.Server Started @1244783ms
[junit4] 2> 1244763 INFO (jetty-launcher-3967-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42663}
[junit4] 2> 1244763 INFO (jetty-launcher-3967-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41719}
[junit4] 2> 1244763 ERROR (jetty-launcher-3967-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1244763 ERROR (jetty-launcher-3967-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1244763 INFO (jetty-launcher-3967-thread-4) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1244763 INFO (jetty-launcher-3967-thread-3) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1244763 INFO (jetty-launcher-3967-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1244763 INFO (jetty-launcher-3967-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1244763 INFO (jetty-launcher-3967-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1244763 INFO (jetty-launcher-3967-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1244763 INFO (jetty-launcher-3967-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1244763 INFO (jetty-launcher-3967-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-13T23:15:18.640771Z
[junit4] 2> 1244763 INFO (jetty-launcher-3967-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1244763 INFO (jetty-launcher-3967-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-13T23:15:18.640800Z
[junit4] 2> 1244764 INFO (zkConnectionManagerCallback-3973-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1244764 INFO (zkConnectionManagerCallback-3974-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1244764 INFO (jetty-launcher-3967-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1244764 INFO (zkConnectionManagerCallback-3978-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1244764 INFO (zkConnectionManagerCallback-3977-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1244764 INFO (jetty-launcher-3967-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1244764 INFO (jetty-launcher-3967-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1244764 INFO (jetty-launcher-3967-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1244767 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1007def3b090002, likely client has closed socket
[junit4] 2> 1244767 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1007def3b090001, likely client has closed socket
[junit4] 2> 1244767 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1007def3b090004, likely client has closed socket
[junit4] 2> 1244812 INFO (jetty-launcher-3967-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40559/solr
[junit4] 2> 1244812 INFO (zkConnectionManagerCallback-3982-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1244822 INFO (zkConnectionManagerCallback-3984-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1244836 INFO (jetty-launcher-3967-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40559/solr
[junit4] 2> 1244836 INFO (zkConnectionManagerCallback-3990-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1244837 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1007def3b090007, likely client has closed socket
[junit4] 2> 1244838 INFO (zkConnectionManagerCallback-3992-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1244844 INFO (jetty-launcher-3967-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40559/solr
[junit4] 2> 1244844 INFO (jetty-launcher-3967-thread-3) [n:127.0.0.1:41719_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1244845 INFO (jetty-launcher-3967-thread-3) [n:127.0.0.1:41719_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41719_solr
[junit4] 2> 1244845 INFO (zkConnectionManagerCallback-3998-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1244845 INFO (jetty-launcher-3967-thread-3) [n:127.0.0.1:41719_solr ] o.a.s.c.Overseer Overseer (id=72196060479029256-127.0.0.1:41719_solr-n_0000000000) starting
[junit4] 2> 1244846 INFO (zkConnectionManagerCallback-4000-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1244849 INFO (zkConnectionManagerCallback-4007-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1244850 INFO (jetty-launcher-3967-thread-3) [n:127.0.0.1:41719_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40559/solr ready
[junit4] 2> 1244850 INFO (jetty-launcher-3967-thread-2) [n:127.0.0.1:35091_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1244851 INFO (jetty-launcher-3967-thread-2) [n:127.0.0.1:35091_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35091_solr
[junit4] 2> 1244857 DEBUG (OverseerAutoScalingTriggerThread-72196060479029256-127.0.0.1:41719_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 1244858 DEBUG (OverseerAutoScalingTriggerThread-72196060479029256-127.0.0.1:41719_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 1244859 DEBUG (OverseerAutoScalingTriggerThread-72196060479029256-127.0.0.1:41719_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 1244859 DEBUG (OverseerAutoScalingTriggerThread-72196060479029256-127.0.0.1:41719_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 1244859 INFO (jetty-launcher-3967-thread-3) [n:127.0.0.1:41719_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1244859 INFO (jetty-launcher-3967-thread-3) [n:127.0.0.1:41719_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41719_solr
[junit4] 2> 1244874 INFO (jetty-launcher-3967-thread-1) [n:127.0.0.1:36443_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1244875 INFO (jetty-launcher-3967-thread-1) [n:127.0.0.1:36443_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1244875 INFO (jetty-launcher-3967-thread-1) [n:127.0.0.1:36443_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1244876 INFO (jetty-launcher-3967-thread-1) [n:127.0.0.1:36443_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36443_solr
[junit4] 2> 1244894 INFO (OverseerStateUpdate-72196060479029256-127.0.0.1:41719_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1244895 INFO (zkCallback-3999-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1244896 INFO (zkCallback-3999-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1244896 INFO (jetty-launcher-3967-thread-3) [n:127.0.0.1:41719_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 1244897 INFO (zkCallback-3983-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1244897 INFO (zkCallback-4006-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1244902 DEBUG (OverseerAutoScalingTriggerThread-72196060479029256-127.0.0.1:41719_solr-n_0000000000) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:35091_solr, 127.0.0.1:41719_solr, 127.0.0.1:36443_solr]
[junit4] 2> 1244906 INFO (zkConnectionManagerCallback-4013-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1244907 DEBUG (OverseerAutoScalingTriggerThread-72196060479029256-127.0.0.1:41719_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1244907 DEBUG (OverseerAutoScalingTriggerThread-72196060479029256-127.0.0.1:41719_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 1244907 INFO (jetty-launcher-3967-thread-2) [n:127.0.0.1:35091_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1244907 DEBUG (ScheduledTrigger-5392-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1244907 INFO (jetty-launcher-3967-thread-2) [n:127.0.0.1:35091_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40559/solr ready
[junit4] 2> 1244911 INFO (jetty-launcher-3967-thread-3) [n:127.0.0.1:41719_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41719.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1bce8115
[junit4] 2> 1244916 INFO (jetty-launcher-3967-thread-3) [n:127.0.0.1:41719_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41719.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1bce8115
[junit4] 2> 1244916 INFO (jetty-launcher-3967-thread-3) [n:127.0.0.1:41719_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41719.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1bce8115
[junit4] 2> 1244917 INFO (jetty-launcher-3967-thread-3) [n:127.0.0.1:41719_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudConsistency_8FF9B23B57A9E206-001/tempDir-001/node3/.
[junit4] 2> 1244918 INFO (jetty-launcher-3967-thread-2) [n:127.0.0.1:35091_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 1244922 INFO (zkConnectionManagerCallback-4019-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1244923 INFO (jetty-launcher-3967-thread-1) [n:127.0.0.1:36443_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1244924 INFO (jetty-launcher-3967-thread-1) [n:127.0.0.1:36443_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40559/solr ready
[junit4] 2> 1244925 INFO (jetty-launcher-3967-thread-1) [n:127.0.0.1:36443_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 1244926 INFO (jetty-launcher-3967-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40559/solr
[junit4] 2> 1244927 INFO (jetty-launcher-3967-thread-2) [n:127.0.0.1:35091_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35091.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1bce8115
[junit4] 2> 1244929 INFO (zkConnectionManagerCallback-4024-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1244931 INFO (zkConnectionManagerCallback-4026-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1244934 INFO (jetty-launcher-3967-thread-4) [n:127.0.0.1:42663_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1244934 INFO (jetty-launcher-3967-thread-1) [n:127.0.0.1:36443_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36443.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1bce8115
[junit4] 2> 1244934 INFO (jetty-launcher-3967-thread-2) [n:127.0.0.1:35091_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35091.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1bce8115
[junit4] 2> 1244934 INFO (jetty-launcher-3967-thread-2) [n:127.0.0.1:35091_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35091.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1bce8115
[junit4] 2> 1244935 INFO (jetty-launcher-3967-thread-4) [n:127.0.0.1:42663_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1244935 INFO (jetty-launcher-3967-thread-2) [n:127.0.0.1:35091_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudConsistency_8FF9B23B57A9E206-001/tempDir-001/node2/.
[junit4] 2> 1244936 INFO (jetty-launcher-3967-thread-4) [n:127.0.0.1:42663_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1244936 INFO (jetty-launcher-3967-thread-4) [n:127.0.0.1:42663_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42663_solr
[junit4] 2> 1244936 INFO (zkCallback-3999-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1244936 INFO (zkCallback-3983-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1244936 INFO (zkCallback-3991-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1244941 INFO (jetty-launcher-3967-thread-1) [n:127.0.0.1:36443_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36443.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1bce8115
[junit4] 2> 1244941 INFO (jetty-launcher-3967-thread-1) [n:127.0.0.1:36443_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36443.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1bce8115
[junit4] 2> 1244942 INFO (jetty-launcher-3967-thread-1) [n:127.0.0.1:36443_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudConsistency_8FF9B23B57A9E206-001/tempDir-001/node1/.
[junit4] 2> 1244942 INFO (zkCallback-4006-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1244943 INFO (zkCallback-4025-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1244943 INFO (zkCallback-4018-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1244947 INFO (zkCallback-4012-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1244949 INFO (zkConnectionManagerCallback-4033-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1244949 INFO (jetty-launcher-3967-thread-4) [n:127.0.0.1:42663_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1244950 INFO (jetty-launcher-3967-thread-4) [n:127.0.0.1:42663_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40559/solr ready
[junit4] 2> 1244950 INFO (jetty-launcher-3967-thread-4) [n:127.0.0.1:42663_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 1244963 INFO (jetty-launcher-3967-thread-4) [n:127.0.0.1:42663_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42663.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1bce8115
[junit4] 2> 1244972 INFO (jetty-launcher-3967-thread-4) [n:127.0.0.1:42663_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42663.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1bce8115
[junit4] 2> 1244972 INFO (jetty-launcher-3967-thread-4) [n:127.0.0.1:42663_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42663.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1bce8115
[junit4] 2> 1244973 INFO (jetty-launcher-3967-thread-4) [n:127.0.0.1:42663_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudConsistency_8FF9B23B57A9E206-001/tempDir-001/node4/.
[junit4] 2> 1245001 INFO (zkConnectionManagerCallback-4036-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1245003 INFO (zkConnectionManagerCallback-4041-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1245003 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1245004 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40559/solr ready
[junit4] 2> 1245007 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@641a6f21{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 1245007 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1747850721
[junit4] 2> 1245008 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1245008 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@55e4765: rootName = solr_41719, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@7a7bc16a
[junit4] 2> 1245012 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1245012 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@714193fa: rootName = solr_41719, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@6b159b5b
[junit4] 2> 1245015 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1245015 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@348b8986: rootName = solr_41719, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@6d50b5e
[junit4] 2> 1245016 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:41719_solr
[junit4] 2> 1245017 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1245017 INFO (zkCallback-3983-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245017 INFO (zkCallback-4006-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245017 INFO (zkCallback-3999-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245017 INFO (zkCallback-4018-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245017 INFO (zkCallback-4025-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245017 INFO (zkCallback-3991-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245018 INFO (zkCallback-4012-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245018 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.Overseer Overseer (id=72196060479029256-127.0.0.1:41719_solr-n_0000000000) closing
[junit4] 2> 1245019 INFO (OverseerStateUpdate-72196060479029256-127.0.0.1:41719_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:41719_solr
[junit4] 2> 1245019 INFO (zkCallback-4032-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245019 INFO (zkCallback-4040-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245019 WARN (OverseerAutoScalingTriggerThread-72196060479029256-127.0.0.1:41719_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1245020 DEBUG (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 1245021 DEBUG (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 1245021 DEBUG (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 1245021 DEBUG (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 1245021 DEBUG (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 1245021 DEBUG (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1245022 INFO (zkCallback-3999-thread-2) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:35091_solr
[junit4] 2> 1245022 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@62913173{/solr,null,UNAVAILABLE}
[junit4] 2> 1245022 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 1245023 INFO (zkCallback-3999-thread-2) [ ] o.a.s.c.Overseer Overseer (id=72196060479029258-127.0.0.1:35091_solr-n_0000000001) starting
[junit4] 2> 1245023 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 10+46
[junit4] 2> 1245024 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1245024 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1245024 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 1245024 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@13a3fc4c{/solr,null,AVAILABLE}
[junit4] 2> 1245025 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.AbstractConnector Started ServerConnector@116c2da4{SSL,[ssl, http/1.1]}{127.0.0.1:43569}
[junit4] 2> 1245025 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.Server Started @1245045ms
[junit4] 2> 1245025 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33057}
[junit4] 2> 1245025 DEBUG (OverseerAutoScalingTriggerThread-72196060479029258-127.0.0.1:35091_solr-n_0000000001) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 1245025 ERROR (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1245025 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1245025 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1245025 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1245025 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1245025 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-13T23:15:18.902934Z
[junit4] 2> 1245026 DEBUG (OverseerAutoScalingTriggerThread-72196060479029258-127.0.0.1:35091_solr-n_0000000001) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 1245026 INFO (zkConnectionManagerCallback-4043-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1245026 DEBUG (OverseerAutoScalingTriggerThread-72196060479029258-127.0.0.1:35091_solr-n_0000000001) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 1245027 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1245027 DEBUG (OverseerAutoScalingTriggerThread-72196060479029258-127.0.0.1:35091_solr-n_0000000001) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:35091_solr, 127.0.0.1:42663_solr, 127.0.0.1:36443_solr]
[junit4] 2> 1245027 DEBUG (OverseerAutoScalingTriggerThread-72196060479029258-127.0.0.1:35091_solr-n_0000000001) [ ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:41719_solr
[junit4] 2> 1245027 DEBUG (OverseerAutoScalingTriggerThread-72196060479029258-127.0.0.1:35091_solr-n_0000000001) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1245027 DEBUG (OverseerAutoScalingTriggerThread-72196060479029258-127.0.0.1:35091_solr-n_0000000001) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 1245028 DEBUG (ScheduledTrigger-5416-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1245028 DEBUG (ScheduledTrigger-5416-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:41719_solr
[junit4] 2> 1245104 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40559/solr
[junit4] 2> 1245105 INFO (zkConnectionManagerCallback-4047-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1245119 INFO (zkConnectionManagerCallback-4049-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1245122 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:33057_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1245123 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:33057_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1245124 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:33057_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1245124 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:33057_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33057_solr
[junit4] 2> 1245125 INFO (zkCallback-4040-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245125 INFO (zkCallback-4018-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245125 INFO (zkCallback-4032-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245125 INFO (zkCallback-4012-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245125 INFO (zkCallback-4025-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245125 INFO (zkCallback-3983-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245125 INFO (zkCallback-3999-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245125 INFO (zkCallback-4048-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245134 INFO (zkConnectionManagerCallback-4056-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1245134 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:33057_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1245134 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:33057_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40559/solr ready
[junit4] 2> 1245135 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:33057_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 1245143 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:33057_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33057.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1bce8115
[junit4] 2> 1245149 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:33057_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33057.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1bce8115
[junit4] 2> 1245149 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:33057_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33057.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1bce8115
[junit4] 2> 1245150 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:33057_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudConsistency_8FF9B23B57A9E206-001/tempDir-001/node3/.
[junit4] 2> 1245185 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.TestCloudConsistency Adding proxy for URL: https://127.0.0.1:43569/solr. Proxy: https://127.0.0.1:33057/solr
[junit4] 2> 1245186 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@30d84aea{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 1245186 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=828955772
[junit4] 2> 1245187 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1245187 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2714cca2: rootName = solr_35091, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@74e103c2
[junit4] 2> 1245194 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1245194 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@765db12d: rootName = solr_35091, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@6b159b5b
[junit4] 2> 1245198 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1245199 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7d6b998b: rootName = solr_35091, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@6d50b5e
[junit4] 2> 1245200 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:35091_solr
[junit4] 2> 1245200 INFO (zkCallback-4032-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245201 INFO (zkCallback-4018-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245201 INFO (zkCallback-3983-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245200 INFO (zkCallback-4040-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245201 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1245201 INFO (zkCallback-4012-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245201 INFO (zkCallback-4048-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245201 INFO (zkCallback-4025-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245201 INFO (zkCallback-3999-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245201 INFO (zkCallback-4055-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245201 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1245201 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.Overseer Overseer (id=72196060479029258-127.0.0.1:35091_solr-n_0000000001) closing
[junit4] 2> 1245201 INFO (OverseerStateUpdate-72196060479029258-127.0.0.1:35091_solr-n_0000000001) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:35091_solr
[junit4] 2> 1245201 WARN (OverseerAutoScalingTriggerThread-72196060479029258-127.0.0.1:35091_solr-n_0000000001) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1245203 DEBUG (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 1245203 DEBUG (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 1245203 DEBUG (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 1245203 DEBUG (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 1245203 DEBUG (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 1245203 DEBUG (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1245204 INFO (zkCallback-3983-thread-1) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36443_solr
[junit4] 2> 1245205 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5ef3d631{/solr,null,UNAVAILABLE}
[junit4] 2> 1245205 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 1245205 INFO (zkCallback-3983-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72196060479029254-127.0.0.1:36443_solr-n_0000000002) starting
[junit4] 2> 1245206 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 10+46
[junit4] 2> 1245206 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1245206 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1245206 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 1245206 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7329585d{/solr,null,AVAILABLE}
[junit4] 2> 1245207 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.AbstractConnector Started ServerConnector@103dbb77{SSL,[ssl, http/1.1]}{127.0.0.1:38159}
[junit4] 2> 1245207 DEBUG (OverseerAutoScalingTriggerThread-72196060479029254-127.0.0.1:36443_solr-n_0000000002) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 1245207 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.Server Started @1245227ms
[junit4] 2> 1245207 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35021}
[junit4] 2> 1245207 DEBUG (OverseerAutoScalingTriggerThread-72196060479029254-127.0.0.1:36443_solr-n_0000000002) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 1245207 DEBUG (OverseerAutoScalingTriggerThread-72196060479029254-127.0.0.1:36443_solr-n_0000000002) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 1245207 ERROR (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1245208 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1245208 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1245208 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1245208 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1245208 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-13T23:15:19.085313Z
[junit4] 2> 1245208 DEBUG (OverseerAutoScalingTriggerThread-72196060479029254-127.0.0.1:36443_solr-n_0000000002) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:33057_solr, 127.0.0.1:42663_solr, 127.0.0.1:36443_solr]
[junit4] 2> 1245208 DEBUG (OverseerAutoScalingTriggerThread-72196060479029254-127.0.0.1:36443_solr-n_0000000002) [ ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:35091_solr
[junit4] 2> 1245209 INFO (zkConnectionManagerCallback-4059-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1245209 DEBUG (OverseerAutoScalingTriggerThread-72196060479029254-127.0.0.1:36443_solr-n_0000000002) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1245209 DEBUG (OverseerAutoScalingTriggerThread-72196060479029254-127.0.0.1:36443_solr-n_0000000002) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 1245209 DEBUG (ScheduledTrigger-5430-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1245209 DEBUG (ScheduledTrigger-5430-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:35091_solr
[junit4] 2> 1245209 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1245343 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40559/solr
[junit4] 2> 1245344 INFO (zkConnectionManagerCallback-4063-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1245346 INFO (zkConnectionManagerCallback-4065-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1245350 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:35021_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1245351 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:35021_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1245352 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:35021_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1245352 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:35021_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35021_solr
[junit4] 2> 1245353 INFO (zkCallback-4055-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245353 INFO (zkCallback-3983-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245353 INFO (zkCallback-4032-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245353 INFO (zkCallback-4025-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245353 INFO (zkCallback-4040-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245354 INFO (zkCallback-4048-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245355 INFO (zkCallback-4018-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245359 INFO (zkCallback-4064-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245367 INFO (zkConnectionManagerCallback-4072-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1245368 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:35021_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1245368 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:35021_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40559/solr ready
[junit4] 2> 1245368 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:35021_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 1245382 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:35021_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35021.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1bce8115
[junit4] 2> 1245393 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:35021_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35021.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1bce8115
[junit4] 2> 1245393 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:35021_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35021.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1bce8115
[junit4] 2> 1245394 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:35021_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudConsistency_8FF9B23B57A9E206-001/tempDir-001/node2/.
[junit4] 2> 1245422 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.TestCloudConsistency Adding proxy for URL: https://127.0.0.1:38159/solr. Proxy: https://127.0.0.1:35021/solr
[junit4] 2> 1245422 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@25159a64{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 1245423 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1848657533
[junit4] 2> 1245423 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1245423 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4ee1a45f: rootName = solr_36443, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@24ea0750
[junit4] 2> 1245428 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1245428 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@318bb5e6: rootName = solr_36443, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@6b159b5b
[junit4] 2> 1245430 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1245430 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@53d5cede: rootName = solr_36443, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@6d50b5e
[junit4] 2> 1245432 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:36443_solr
[junit4] 2> 1245432 INFO (zkCallback-4032-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245432 INFO (zkCallback-4048-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245432 INFO (zkCallback-4040-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245432 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1245432 INFO (zkCallback-4018-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245433 INFO (zkCallback-4025-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245432 INFO (zkCallback-4055-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245433 INFO (zkCallback-4064-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245432 INFO (zkCallback-3983-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245433 INFO (zkCallback-4071-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245433 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.Overseer Overseer (id=72196060479029254-127.0.0.1:36443_solr-n_0000000002) closing
[junit4] 2> 1245433 INFO (OverseerStateUpdate-72196060479029254-127.0.0.1:36443_solr-n_0000000002) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:36443_solr
[junit4] 2> 1245433 WARN (OverseerAutoScalingTriggerThread-72196060479029254-127.0.0.1:36443_solr-n_0000000002) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1245434 DEBUG (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 1245434 DEBUG (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 1245434 DEBUG (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 1245434 DEBUG (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 1245434 DEBUG (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 1245434 DEBUG (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1245435 INFO (zkCallback-4025-thread-1) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42663_solr
[junit4] 2> 1245435 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3c16ae9a{/solr,null,UNAVAILABLE}
[junit4] 2> 1245435 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 1245435 INFO (zkCallback-4025-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72196060479029263-127.0.0.1:42663_solr-n_0000000003) starting
[junit4] 2> 1245436 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 10+46
[junit4] 2> 1245437 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1245437 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1245437 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 1245437 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2f5f3675{/solr,null,AVAILABLE}
[junit4] 2> 1245437 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.AbstractConnector Started ServerConnector@21e3ef01{SSL,[ssl, http/1.1]}{127.0.0.1:37707}
[junit4] 2> 1245437 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.Server Started @1245457ms
[junit4] 2> 1245438 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37457}
[junit4] 2> 1245438 ERROR (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1245438 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1245438 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1245438 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1245438 DEBUG (OverseerAutoScalingTriggerThread-72196060479029263-127.0.0.1:42663_solr-n_0000000003) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 1245438 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1245438 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-13T23:15:19.315542Z
[junit4] 2> 1245438 DEBUG (OverseerAutoScalingTriggerThread-72196060479029263-127.0.0.1:42663_solr-n_0000000003) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 1245438 DEBUG (OverseerAutoScalingTriggerThread-72196060479029263-127.0.0.1:42663_solr-n_0000000003) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 1245439 INFO (zkConnectionManagerCallback-4075-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1245439 DEBUG (OverseerAutoScalingTriggerThread-72196060479029263-127.0.0.1:42663_solr-n_0000000003) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:35021_solr, 127.0.0.1:33057_solr, 127.0.0.1:42663_solr]
[junit4] 2> 1245439 DEBUG (OverseerAutoScalingTriggerThread-72196060479029263-127.0.0.1:42663_solr-n_0000000003) [ ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:36443_solr
[junit4] 2> 1245439 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1245439 DEBUG (OverseerAutoScalingTriggerThread-72196060479029263-127.0.0.1:42663_solr-n_0000000003) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1245439 DEBUG (OverseerAutoScalingTriggerThread-72196060479029263-127.0.0.1:42663_solr-n_0000000003) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 1245439 DEBUG (ScheduledTrigger-5444-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1245439 DEBUG (ScheduledTrigger-5444-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:36443_solr
[junit4] 2> 1245514 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40559/solr
[junit4] 2> 1245515 INFO (zkConnectionManagerCallback-4079-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1245516 INFO (zkConnectionManagerCallback-4081-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1245519 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:37457_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1245520 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:37457_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1245520 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:37457_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1245520 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:37457_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37457_solr
[junit4] 2> 1245521 INFO (zkCallback-4040-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245521 INFO (zkCallback-4071-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245521 INFO (zkCallback-4048-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245521 INFO (zkCallback-4025-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245521 INFO (zkCallback-4064-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245521 INFO (zkCallback-4032-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245521 INFO (zkCallback-4055-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245522 INFO (zkCallback-4080-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245550 INFO (zkConnectionManagerCallback-4088-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1245550 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:37457_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1245551 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:37457_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40559/solr ready
[junit4] 2> 1245551 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:37457_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 1245560 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:37457_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37457.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1bce8115
[junit4] 2> 1245566 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:37457_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37457.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1bce8115
[junit4] 2> 1245566 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:37457_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37457.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1bce8115
[junit4] 2> 1245567 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:37457_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudConsistency_8FF9B23B57A9E206-001/tempDir-001/node1/.
[junit4] 2> 1245592 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.TestCloudConsistency Adding proxy for URL: https://127.0.0.1:37707/solr. Proxy: https://127.0.0.1:37457/solr
[junit4] 2> 1245593 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@109f300b{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 1245593 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1608369294
[junit4] 2> 1245593 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1245593 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3484668a: rootName = solr_42663, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@5e70d091
[junit4] 2> 1245598 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1245598 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@446283: rootName = solr_42663, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@6b159b5b
[junit4] 2> 1245600 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1245600 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7f9e8d1d: rootName = solr_42663, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@6d50b5e
[junit4] 2> 1245601 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:42663_solr
[junit4] 2> 1245602 INFO (zkCallback-4040-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245602 INFO (zkCallback-4055-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245602 INFO (zkCallback-4032-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245602 INFO (zkCallback-4071-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245602 INFO (zkCallback-4048-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245602 INFO (zkCallback-4025-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245602 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1245602 INFO (zkCallback-4064-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245602 INFO (zkCallback-4087-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245602 INFO (zkCallback-4080-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1245603 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.Overseer Overseer (id=72196060479029263-127.0.0.1:42663_solr-n_0000000003) closing
[junit4] 2> 1245603 INFO (OverseerStateUpdate-72196060479029263-127.0.0.1:42663_solr-n_0000000003) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42663_solr
[junit4] 2> 1245603 WARN (OverseerAutoScalingTriggerThread-72196060479029263-127.0.0.1:42663_solr-n_0000000003) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1245604 DEBUG (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 1245604 DEBUG (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 1245604 DEBUG (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 1245604 DEBUG (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 1245604 DEBUG (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 1245604 DEBUG (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1245605 INFO (zkCallback-4048-thread-1) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33057_solr
[junit4] 2> 1245605 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1d6e67f5{/solr,null,UNAVAILABLE}
[junit4] 2> 1245605 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 1245605 INFO (zkCallback-4048-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72196060479029269-127.0.0.1:33057_solr-n_0000000004) starting
[junit4] 2> 1245606 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 10+46
[junit4] 2> 1245607 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1245607 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1245607 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 1245607 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@464ff27{/solr,null,AVAILABLE}
[junit4] 2> 1245608 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.AbstractConnector Started ServerConnector@42c001c5{SSL,[ssl, http/1.1]}{127.0.0.1:38309}
[junit4] 2> 1245608 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.e.j.s.Server Started @1245628ms
[junit4] 2> 1245608 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45225}
[junit4] 2> 1245608 ERROR (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1245609 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1245609 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1245609 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1245609 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1245609 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-13T23:15:19.486336Z
[junit4] 2> 1245610 INFO (zkConnectionManagerCallback-4091-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1245610 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1245612 DEBUG (OverseerAutoScalingTriggerThread-72196060479029269-127.0.0.1:33057_solr-n_0000000004) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 1245612 DEBUG (OverseerAutoScalingTriggerThread-72196060479029269-127.0.0.1:33057_solr-n_0000000004) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 1245612 DEBUG (OverseerAutoScalingTriggerThread-72196060479029269-127.0.0.1:33057_solr-n_0000000004) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 1245613 DEBUG (OverseerAutoScalingTriggerThread-72196060479029269-127.0.0.1:33057_solr-n_0000000004) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:35021_solr, 127.0.0.1:37457_solr, 127.0.0.1:33057_solr]
[junit4] 2> 1245613 DEBUG (OverseerAutoScalingTriggerThread-72196060479029269-127.0.0.1:33057_solr-n_0000000004) [ ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:42663_solr
[junit4] 2> 1245613 DEBUG (OverseerAutoScalingTriggerThread-72196060479029269-127.0.0.1:33057_solr-n_0000000004) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1245613 DEBUG (OverseerAutoScalingTriggerThread-72196060479029269-127.0.0.1:33057_solr-n_0000000004) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 1245613 DEBUG (ScheduledTrigger-5458-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1245613 DEBUG (ScheduledTrigger-5458-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:42663_solr
[junit4] 2> 1245658 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40559/solr
[junit4] 2> 1245659 INFO (zkConnectionManagerCallback-4095-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1245660 INFO (zkConnectionManagerCallback-4097-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1245663 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:45225_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1245664 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:45225_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1245664 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:45225_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1245664 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:45225_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45225_solr
[junit4] 2> 1245665 INFO (zkCallback-4055-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245665 INFO (zkCallback-4048-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245665 INFO (zkCallback-4040-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245665 INFO (zkCallback-4064-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245665 INFO (zkCallback-4071-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245665 INFO (zkCallback-4080-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245665 INFO (zkCallback-4087-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245665 INFO (zkCallback-4096-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1245679 INFO (zkConnectionManagerCallback-4104-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1245680 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:45225_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1245680 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:45225_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40559/solr ready
[junit4] 2> 1245680 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:45225_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 1245689 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [n:127.0.0.1:45225_solr ] o.a.s.m.r.SolrJmxReporter JMX
[...truncated too long message...]
3968-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@136576de{/solr,null,UNAVAILABLE}
[junit4] 2> 1294075 INFO (jetty-closer-3968-thread-3) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 1294078 INFO (coreCloseExecutor-5590-thread-1) [n:127.0.0.1:33057_solr c:outOfSyncReplicasCannotBecomeLeader-false s:shard1 r:core_node62 x:outOfSyncReplicasCannotBecomeLeader-false_shard1_replica_n61] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.outOfSyncReplicasCannotBecomeLeader-false.shard1.leader, tag=6ed857b8
[junit4] 2> 1294080 INFO (jetty-closer-3968-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@58445a1a{/solr,null,UNAVAILABLE}
[junit4] 2> 1294080 INFO (jetty-closer-3968-thread-1) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 1295571 INFO (jetty-closer-3968-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2f4b2cc2{/solr,null,UNAVAILABLE}
[junit4] 2> 1295571 INFO (jetty-closer-3968-thread-2) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 1295572 ERROR (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1295572 INFO (SUITE-TestCloudConsistency-seed#[8FF9B23B57A9E206]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40559 40559
[junit4] 2> 1295617 INFO (Thread-4530) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40559 40559
[junit4] 2> 1295617 WARN (Thread-4530) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 48 /solr/collections/outOfSyncReplicasCannotBecomeLeader-false/terms/shard1
[junit4] 2> 45 /solr/collections/outOfSyncReplicasCannotBecomeLeader-true/terms/shard1
[junit4] 2> 25 /solr/aliases.json
[junit4] 2> 12 /solr/security.json
[junit4] 2> 7 /solr/configs/outOfSyncReplicasCannotBecomeLeader-false.AUTOCREATED/managed-schema
[junit4] 2> 7 /solr/configs/outOfSyncReplicasCannotBecomeLeader-false.AUTOCREATED
[junit4] 2> 6 /solr/configs/outOfSyncReplicasCannotBecomeLeader-true.AUTOCREATED
[junit4] 2> 6 /solr/configs/outOfSyncReplicasCannotBecomeLeader-true.AUTOCREATED/managed-schema
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 43 /solr/collections/outOfSyncReplicasCannotBecomeLeader-false/state.json
[junit4] 2> 32 /solr/collections/outOfSyncReplicasCannotBecomeLeader-true/state.json
[junit4] 2> 25 /solr/clusterprops.json
[junit4] 2> 25 /solr/clusterstate.json
[junit4] 2> 7 /solr/autoscaling.json
[junit4] 2> 2 /solr/overseer_elect/election/72196060479029281-127.0.0.1:45225_solr-n_0000000007
[junit4] 2> 2 /solr/collections/outOfSyncReplicasCannotBecomeLeader-true/leader_elect/shard1/election/72196060479029285-core_node62-n_0000000000
[junit4] 2> 2 /solr/collections/outOfSyncReplicasCannotBecomeLeader-false/leader_elect/shard1/election/72196060479029285-core_node62-n_0000000003
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 25 /solr/live_nodes
[junit4] 2> 25 /solr/collections
[junit4] 2> 7 /solr/overseer/queue
[junit4] 2> 7 /solr/autoscaling/events/.scheduled_maintenance
[junit4] 2> 7 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 7 /solr/overseer/collection-queue-work
[junit4] 2> 7 /solr/overseer/queue-work
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestCloudConsistency_8FF9B23B57A9E206-001
[junit4] 2> Jun 13, 2018 11:16:09 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=3803, maxDocsPerChunk=5, blockSize=1), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=3803, blockSize=1)), sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@3b7d46c8), locale=eo-001, timezone=Etc/GMT+2
[junit4] 2> NOTE: Linux 4.13.0-41-generic amd64/Oracle Corporation 10 (64-bit)/cpus=8,threads=1,free=149359720,total=518979584
[junit4] 2> NOTE: All tests run in this JVM: [TestObjectReleaseTracker, DirectUpdateHandlerTest, DeleteShardTest, DistributedQueryElevationComponentTest, AsyncCallRequestStatusResponseTest, TestNumericRangeQuery32, TestRestManager, TestHdfsBackupRestoreCore, SolrCoreCheckLockOnStartupTest, SolrCoreMetricManagerTest, TestCloudJSONFacetSKG, TestDistributedSearch, TestClusterStateMutator, DistribDocExpirationUpdateProcessorTest, StatsComponentTest, ShufflingReplicaListTransformerTest, DistributedFacetPivotSmallAdvancedTest, DistributedQueryComponentCustomSortTest, HdfsUnloadDistributedZkTest, TestLRUCache, TestSolrJ, SuggesterWFSTTest, ExternalFileFieldSortTest, CoreMergeIndexesAdminHandlerTest, AssignTest, TestConfigSetsAPI, HdfsDirectoryTest, TestMergePolicyConfig, TestLeaderElectionWithEmptyReplica, TestConfigSetProperties, TestDocBasedVersionConstraints, TestManagedStopFilterFactory, FileUtilsTest, ParsingFieldUpdateProcessorsTest, TestStressCloudBlindAtomicUpdates, TestCloudDeleteByQuery, RegexBytesRefFilterTest, ConfigureRecoveryStrategyTest, XmlInterpolationTest, TestWordDelimiterFilterFactory, TestMinMaxOnMultiValuedField, SolrRequestParserTest, HighlighterConfigTest, HdfsRecoverLeaseTest, TestJoin, MetricUtilsTest, CursorPagingTest, TestSolrCloudWithDelegationTokens, TestDocTermOrdsUninvertLimit, HealthCheckHandlerTest, TestNestedDocsSort, BlobRepositoryCloudTest, ScriptEngineTest, TestSort, MoveReplicaTest, TestHashPartitioner, SpellCheckCollatorTest, MaxSizeAutoCommitTest, AnalyticsQueryTest, TestExecutePlanAction, RecoveryAfterSoftCommitTest, TestHttpShardHandlerFactory, BadIndexSchemaTest, EchoParamsTest, SmileWriterTest, TestJsonFacetRefinement, TestRuleBasedAuthorizationPlugin, TestCustomStream, TemplateUpdateProcessorTest, SolrMetricsIntegrationTest, CdcrReplicationHandlerTest, ZkSolrClientTest, TestImpersonationWithHadoopAuth, TestSolr4Spatial2, DefaultValueUpdateProcessorTest, V2ApiIntegrationTest, ExitableDirectoryReaderTest, TriggerCooldownIntegrationTest, TestBlobHandler, BadCopyFieldTest, BJQParserTest, TestDynamicFieldCollectionResource, TestReRankQParserPlugin, TestFileDictionaryLookup, TestTestInjection, TestQuerySenderNoQuery, TestMaxScoreQueryParser, TestCollationField, TestCloudRecovery, CollectionsAPIAsyncDistributedZkTest, TestSimDistributedQueue, TestFieldCacheWithThreads, TestDistribIDF, HdfsAutoAddReplicasIntegrationTest, TestDynamicURP, TestLRUStatsCache, TestCryptoKeys, CheckHdfsIndexTest, XmlUpdateRequestHandlerTest, ShardRoutingTest, SortSpecParsingTest, PKIAuthenticationIntegrationTest, AnalysisErrorHandlingTest, HLLSerializationTest, TestClusterStateProvider, DocExpirationUpdateProcessorFactoryTest, DocumentAnalysisRequestHandlerTest, TermVectorComponentDistributedTest, OverseerCollectionConfigSetProcessorTest, TestPrepRecovery, TestExportWriter, TestMacroExpander, OverseerStatusTest, TestUtilizeNode, TestOmitPositions, TestRemoteStreaming, HttpSolrCallGetCoreTest, AddReplicaTest, NodeMarkersRegistrationTest, AutoScalingHandlerTest, MissingSegmentRecoveryTest, SchemaApiFailureTest, TestStandardQParsers, TestXmlQParser, TestHdfsCloudBackupRestore, TestDistributedMap, RAMDirectoryFactoryTest, DistributedTermsComponentTest, TestPseudoReturnFields, SystemInfoHandlerTest, IndexBasedSpellCheckerTest, TestSolrConfigHandlerCloud, BinaryUpdateRequestHandlerTest, TestPerFieldSimilarityWithDefaultOverride, TestUseDocValuesAsStored, HighlighterTest, SoftAutoCommitTest, CloudExitableDirectoryReaderTest, CollectionStateFormat2Test, ConcurrentCreateRoutedAliasTest, ConnectionManagerTest, CreateCollectionCleanupTest, DeleteLastCustomShardedReplicaTest, DeleteStatusTest, LIROnShardRestartTest, LIRRollingUpdatesTest, LeaderFailoverAfterPartitionTest, LegacyCloudClusterPropTest, MetricsHistoryIntegrationTest, SharedFSAutoReplicaFailoverTest, SolrCLIZkUtilsTest, SolrCloudExampleTest, SolrXmlInZkTest, TestCloudConsistency]
[junit4] Completed [593/817 (1!)] on J0 in 51.98s, 2 tests, 1 failure <<< FAILURES!
[...truncated 44684 lines...]
[repro] Jenkins log URL: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/22245/consoleText
[repro] Revision: eea4197a3aa3b1fffa31a2708e0947aabd7e727b
[repro] Ant options: "-Dargs=-XX:+UseCompressedOops -XX:+UseSerialGC"
[repro] Repro line: ant test -Dtestcase=TestCloudConsistency -Dtests.method=testOutOfSyncReplicasCannotBecomeLeader -Dtests.seed=8FF9B23B57A9E206 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=eo-001 -Dtests.timezone=Etc/GMT+2 -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[repro] ant clean
[...truncated 6 lines...]
[repro] Test suites by module:
[repro] solr/core
[repro] TestCloudConsistency
[repro] ant compile-test
[...truncated 3305 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.TestCloudConsistency" -Dtests.showOutput=onerror "-Dargs=-XX:+UseCompressedOops -XX:+UseSerialGC" -Dtests.seed=8FF9B23B57A9E206 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=eo-001 -Dtests.timezone=Etc/GMT+2 -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[...truncated 81 lines...]
[repro] Failures:
[repro] 0/5 failed: org.apache.solr.cloud.TestCloudConsistency
[repro] Exiting with code 0
[...truncated 40 lines...]