You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/11/22 07:25:40 UTC
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1703 - Still
Unstable
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1703/
1 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMixedBounds
Error Message:
expected:<MERGESHARDS> but was:<SPLITSHARD>
Stack Trace:
java.lang.AssertionError: expected:<MERGESHARDS> but was:<SPLITSHARD>
at __randomizedtesting.SeedInfo.seed([E3777760836268B3:E9F4C8CDCED963E9]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:147)
at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMixedBounds(IndexSizeTriggerTest.java:673)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 14957 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E3777760836268B3-001/init-core-data-001
[junit4] 2> 2725796 WARN (SUITE-IndexSizeTriggerTest-seed#[E3777760836268B3]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=60 numCloses=60
[junit4] 2> 2725797 INFO (SUITE-IndexSizeTriggerTest-seed#[E3777760836268B3]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 2725800 INFO (SUITE-IndexSizeTriggerTest-seed#[E3777760836268B3]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 2725801 INFO (SUITE-IndexSizeTriggerTest-seed#[E3777760836268B3]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 2725801 INFO (SUITE-IndexSizeTriggerTest-seed#[E3777760836268B3]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E3777760836268B3-001/tempDir-001
[junit4] 2> 2725801 INFO (SUITE-IndexSizeTriggerTest-seed#[E3777760836268B3]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2725802 INFO (Thread-11153) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2725802 INFO (Thread-11153) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2725808 ERROR (Thread-11153) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2725902 INFO (SUITE-IndexSizeTriggerTest-seed#[E3777760836268B3]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:40318
[junit4] 2> 2725905 INFO (zkConnectionManagerCallback-53154-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2725913 INFO (jetty-launcher-53151-thread-2) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_191-b12
[junit4] 2> 2725913 INFO (jetty-launcher-53151-thread-1) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_191-b12
[junit4] 2> 2725914 INFO (jetty-launcher-53151-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2725914 INFO (jetty-launcher-53151-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2725914 INFO (jetty-launcher-53151-thread-1) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 2725915 INFO (jetty-launcher-53151-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@8158d8{/solr,null,AVAILABLE}
[junit4] 2> 2725916 INFO (jetty-launcher-53151-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2725916 INFO (jetty-launcher-53151-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2725916 INFO (jetty-launcher-53151-thread-2) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 2725916 INFO (jetty-launcher-53151-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@613f1d39{SSL,[ssl, http/1.1]}{127.0.0.1:43052}
[junit4] 2> 2725916 INFO (jetty-launcher-53151-thread-1) [ ] o.e.j.s.Server Started @2725995ms
[junit4] 2> 2725916 INFO (jetty-launcher-53151-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43052}
[junit4] 2> 2725916 ERROR (jetty-launcher-53151-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2725916 INFO (jetty-launcher-53151-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@edacbe3{/solr,null,AVAILABLE}
[junit4] 2> 2725916 INFO (jetty-launcher-53151-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2725916 INFO (jetty-launcher-53151-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 2725916 INFO (jetty-launcher-53151-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2725917 INFO (jetty-launcher-53151-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2725917 INFO (jetty-launcher-53151-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-11-22T06:40:52.020Z
[junit4] 2> 2725917 INFO (jetty-launcher-53151-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@190a02f4{SSL,[ssl, http/1.1]}{127.0.0.1:33543}
[junit4] 2> 2725917 INFO (jetty-launcher-53151-thread-2) [ ] o.e.j.s.Server Started @2725996ms
[junit4] 2> 2725917 INFO (jetty-launcher-53151-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33543}
[junit4] 2> 2725917 ERROR (jetty-launcher-53151-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2725917 INFO (jetty-launcher-53151-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2725917 INFO (jetty-launcher-53151-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 2725917 INFO (jetty-launcher-53151-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2725917 INFO (jetty-launcher-53151-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2725917 INFO (jetty-launcher-53151-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-11-22T06:40:52.020Z
[junit4] 2> 2725920 INFO (zkConnectionManagerCallback-53156-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2725920 INFO (zkConnectionManagerCallback-53158-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2725920 INFO (jetty-launcher-53151-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2725921 INFO (jetty-launcher-53151-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2726087 INFO (jetty-launcher-53151-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40318/solr
[junit4] 2> 2726088 INFO (zkConnectionManagerCallback-53162-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2726091 INFO (zkConnectionManagerCallback-53164-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2726230 INFO (jetty-launcher-53151-thread-1) [n:127.0.0.1:43052_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43052_solr
[junit4] 2> 2726231 INFO (jetty-launcher-53151-thread-1) [n:127.0.0.1:43052_solr ] o.a.s.c.Overseer Overseer (id=72319976252375044-127.0.0.1:43052_solr-n_0000000000) starting
[junit4] 2> 2726240 INFO (zkConnectionManagerCallback-53171-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2726243 INFO (jetty-launcher-53151-thread-1) [n:127.0.0.1:43052_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40318/solr ready
[junit4] 2> 2726244 INFO (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:43052_solr
[junit4] 2> 2726245 INFO (jetty-launcher-53151-thread-1) [n:127.0.0.1:43052_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43052_solr
[junit4] 2> 2726251 DEBUG (OverseerAutoScalingTriggerThread-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 2726252 INFO (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2726253 DEBUG (OverseerAutoScalingTriggerThread-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 2726253 DEBUG (OverseerAutoScalingTriggerThread-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 2726253 DEBUG (OverseerAutoScalingTriggerThread-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 2726254 INFO (zkCallback-53170-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2726265 DEBUG (OverseerAutoScalingTriggerThread-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:43052_solr]
[junit4] 2> 2726267 DEBUG (OverseerAutoScalingTriggerThread-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2726268 DEBUG (OverseerAutoScalingTriggerThread-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 2726269 DEBUG (ScheduledTrigger-13588-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 2726282 INFO (jetty-launcher-53151-thread-1) [n:127.0.0.1:43052_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 2726335 INFO (jetty-launcher-53151-thread-1) [n:127.0.0.1:43052_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43052.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c80d78a
[junit4] 2> 2726345 INFO (jetty-launcher-53151-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40318/solr
[junit4] 2> 2726347 INFO (zkConnectionManagerCallback-53176-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2726348 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100eea29eb00006, likely client has closed socket
[junit4] 2> 2726350 INFO (zkConnectionManagerCallback-53178-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2726351 INFO (jetty-launcher-53151-thread-1) [n:127.0.0.1:43052_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43052.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c80d78a
[junit4] 2> 2726351 INFO (jetty-launcher-53151-thread-1) [n:127.0.0.1:43052_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43052.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c80d78a
[junit4] 2> 2726353 INFO (jetty-launcher-53151-thread-1) [n:127.0.0.1:43052_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E3777760836268B3-001/tempDir-001/node1/.
[junit4] 2> 2726360 INFO (jetty-launcher-53151-thread-2) [n:127.0.0.1:33543_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2726366 INFO (jetty-launcher-53151-thread-2) [n:127.0.0.1:33543_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2726366 INFO (jetty-launcher-53151-thread-2) [n:127.0.0.1:33543_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33543_solr
[junit4] 2> 2726367 DEBUG (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:33543_solr"} current state version: 0
[junit4] 2> 2726368 INFO (zkCallback-53170-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2726368 INFO (zkCallback-53163-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2726372 INFO (zkCallback-53177-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2726415 INFO (zkConnectionManagerCallback-53185-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2726416 INFO (jetty-launcher-53151-thread-2) [n:127.0.0.1:33543_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2726418 INFO (jetty-launcher-53151-thread-2) [n:127.0.0.1:33543_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40318/solr ready
[junit4] 2> 2726418 INFO (jetty-launcher-53151-thread-2) [n:127.0.0.1:33543_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 2726449 INFO (jetty-launcher-53151-thread-2) [n:127.0.0.1:33543_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33543.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c80d78a
[junit4] 2> 2726465 INFO (jetty-launcher-53151-thread-2) [n:127.0.0.1:33543_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33543.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c80d78a
[junit4] 2> 2726465 INFO (jetty-launcher-53151-thread-2) [n:127.0.0.1:33543_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33543.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c80d78a
[junit4] 2> 2726466 INFO (jetty-launcher-53151-thread-2) [n:127.0.0.1:33543_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E3777760836268B3-001/tempDir-001/node2/.
[junit4] 2> 2726512 INFO (zkConnectionManagerCallback-53188-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2726517 INFO (zkConnectionManagerCallback-53193-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2726518 INFO (SUITE-IndexSizeTriggerTest-seed#[E3777760836268B3]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2726520 INFO (SUITE-IndexSizeTriggerTest-seed#[E3777760836268B3]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40318/solr ready
[junit4] 2> 2726558 INFO (TEST-IndexSizeTriggerTest.testMaxOps-seed#[E3777760836268B3]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testMaxOps
[junit4] 2> 2726583 INFO (qtp2122523878-114160) [n:127.0.0.1:43052_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf&maxShardsPerNode=10&name=testMaxOps_collection&nrtReplicas=2&action=CREATE&numShards=5&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2726587 INFO (OverseerThreadFactory-13590-thread-1-processing-n:127.0.0.1:43052_solr) [n:127.0.0.1:43052_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection testMaxOps_collection
[junit4] 2> 2726598 DEBUG (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "name":"testMaxOps_collection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "collection.configName":"conf",
[junit4] 2> "numShards":"5",
[junit4] 2> "maxShardsPerNode":"10",
[junit4] 2> "nrtReplicas":"2",
[junit4] 2> "stateFormat":"2",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "operation":"create"} current state version: 0
[junit4] 2> 2726706 INFO (qtp2122523878-114159) [n:127.0.0.1:43052_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2726734 INFO (qtp628727391-114168) [n:127.0.0.1:33543_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2726750 DEBUG (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testMaxOps_collection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:33543/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 2726750 INFO (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testMaxOps_collection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:33543/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2726754 DEBUG (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testMaxOps_collection_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:43052/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 2726754 INFO (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testMaxOps_collection_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:43052/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2726758 DEBUG (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"testMaxOps_collection_shard2_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:43052/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 2726758 INFO (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"testMaxOps_collection_shard2_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:43052/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2726761 DEBUG (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.Overseer processMessage: queueSize: 4, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"testMaxOps_collection_shard2_replica_n6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:33543/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 2726761 INFO (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"testMaxOps_collection_shard2_replica_n6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:33543/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2726765 DEBUG (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.Overseer processMessage: queueSize: 5, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "shard":"shard3",
[junit4] 2> "core":"testMaxOps_collection_shard3_replica_n8",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:33543/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 2726765 INFO (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "shard":"shard3",
[junit4] 2> "core":"testMaxOps_collection_shard3_replica_n8",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:33543/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2726768 DEBUG (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.Overseer processMessage: queueSize: 6, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "shard":"shard3",
[junit4] 2> "core":"testMaxOps_collection_shard3_replica_n10",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:43052/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 2726768 INFO (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "shard":"shard3",
[junit4] 2> "core":"testMaxOps_collection_shard3_replica_n10",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:43052/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2726773 DEBUG (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.Overseer processMessage: queueSize: 7, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "shard":"shard4",
[junit4] 2> "core":"testMaxOps_collection_shard4_replica_n12",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:43052/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 2726773 INFO (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "shard":"shard4",
[junit4] 2> "core":"testMaxOps_collection_shard4_replica_n12",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:43052/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2726777 DEBUG (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.Overseer processMessage: queueSize: 8, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "shard":"shard4",
[junit4] 2> "core":"testMaxOps_collection_shard4_replica_n14",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:33543/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 2726777 INFO (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "shard":"shard4",
[junit4] 2> "core":"testMaxOps_collection_shard4_replica_n14",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:33543/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2726780 DEBUG (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.Overseer processMessage: queueSize: 9, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "shard":"shard5",
[junit4] 2> "core":"testMaxOps_collection_shard5_replica_n16",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:33543/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 2726780 INFO (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "shard":"shard5",
[junit4] 2> "core":"testMaxOps_collection_shard5_replica_n16",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:33543/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2726788 DEBUG (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.Overseer processMessage: queueSize: 10, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "shard":"shard5",
[junit4] 2> "core":"testMaxOps_collection_shard5_replica_n18",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:43052/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 2726789 INFO (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "shard":"shard5",
[junit4] 2> "core":"testMaxOps_collection_shard5_replica_n18",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:43052/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2726993 INFO (qtp2122523878-114158) [n:127.0.0.1:43052_solr x:testMaxOps_collection_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf&newCollection=true&name=testMaxOps_collection_shard1_replica_n2&action=CREATE&numShards=5&collection=testMaxOps_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2726994 INFO (qtp2122523878-114155) [n:127.0.0.1:43052_solr x:testMaxOps_collection_shard2_replica_n4] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf&newCollection=true&name=testMaxOps_collection_shard2_replica_n4&action=CREATE&numShards=5&collection=testMaxOps_collection&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2726994 INFO (qtp2122523878-114158) [n:127.0.0.1:43052_solr x:testMaxOps_collection_shard1_replica_n2] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2727001 INFO (qtp2122523878-114159) [n:127.0.0.1:43052_solr x:testMaxOps_collection_shard3_replica_n10] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node13&collection.configName=conf&newCollection=true&name=testMaxOps_collection_shard3_replica_n10&action=CREATE&numShards=5&collection=testMaxOps_collection&shard=shard3&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2727001 INFO (qtp628727391-114169) [n:127.0.0.1:33543_solr x:testMaxOps_collection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=testMaxOps_collection_shard1_replica_n1&action=CREATE&numShards=5&collection=testMaxOps_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2727002 INFO (qtp628727391-114165) [n:127.0.0.1:33543_solr x:testMaxOps_collection_shard2_replica_n6] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node9&collection.configName=conf&newCollection=true&name=testMaxOps_collection_shard2_replica_n6&action=CREATE&numShards=5&collection=testMaxOps_collection&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2727003 INFO (qtp628727391-114166) [n:127.0.0.1:33543_solr x:testMaxOps_collection_shard3_replica_n8] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node11&collection.configName=conf&newCollection=true&name=testMaxOps_collection_shard3_replica_n8&action=CREATE&numShards=5&collection=testMaxOps_collection&shard=shard3&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2727004 INFO (qtp2122523878-114161) [n:127.0.0.1:43052_solr x:testMaxOps_collection_shard4_replica_n12] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node15&collection.configName=conf&newCollection=true&name=testMaxOps_collection_shard4_replica_n12&action=CREATE&numShards=5&collection=testMaxOps_collection&shard=shard4&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2727005 INFO (qtp628727391-114167) [n:127.0.0.1:33543_solr x:testMaxOps_collection_shard5_replica_n16] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node19&collection.configName=conf&newCollection=true&name=testMaxOps_collection_shard5_replica_n16&action=CREATE&numShards=5&collection=testMaxOps_collection&shard=shard5&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2727005 INFO (qtp2122523878-114157) [n:127.0.0.1:43052_solr x:testMaxOps_collection_shard5_replica_n18] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node20&collection.configName=conf&newCollection=true&name=testMaxOps_collection_shard5_replica_n18&action=CREATE&numShards=5&collection=testMaxOps_collection&shard=shard5&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2727007 INFO (qtp628727391-114163) [n:127.0.0.1:33543_solr x:testMaxOps_collection_shard4_replica_n14] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node17&collection.configName=conf&newCollection=true&name=testMaxOps_collection_shard4_replica_n14&action=CREATE&numShards=5&collection=testMaxOps_collection&shard=shard4&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2727007 DEBUG (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":"testMaxOps_collection_shard2_replica_n4",
[junit4] 2> "core_node_name":"core_node7",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:43052/solr",
[junit4] 2> "node_name":"127.0.0.1:43052_solr",
[junit4] 2> "numShards":"5",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "force_set_state":"false",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 2727010 DEBUG (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "core":"testMaxOps_collection_shard1_replica_n2",
[junit4] 2> "core_node_name":"core_node5",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:43052/solr",
[junit4] 2> "node_name":"127.0.0.1:43052_solr",
[junit4] 2> "numShards":"5",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "force_set_state":"false",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 2727010 DEBUG (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "core":"testMaxOps_collection_shard3_replica_n10",
[junit4] 2> "core_node_name":"core_node13",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:43052/solr",
[junit4] 2> "node_name":"127.0.0.1:43052_solr",
[junit4] 2> "numShards":"5",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "force_set_state":"false",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 2727015 DEBUG (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.Overseer processMessage: queueSize: 7, message = {
[junit4] 2> "core":"testMaxOps_collection_shard5_replica_n18",
[junit4] 2> "core_node_name":"core_node20",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:43052/solr",
[junit4] 2> "node_name":"127.0.0.1:43052_solr",
[junit4] 2> "numShards":"5",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard5",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "force_set_state":"false",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 2727015 DEBUG (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.Overseer processMessage: queueSize: 7, message = {
[junit4] 2> "core":"testMaxOps_collection_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:33543/solr",
[junit4] 2> "node_name":"127.0.0.1:33543_solr",
[junit4] 2> "numShards":"5",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "force_set_state":"false",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 2727015 DEBUG (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.Overseer processMessage: queueSize: 7, message = {
[junit4] 2> "core":"testMaxOps_collection_shard5_replica_n16",
[junit4] 2> "core_node_name":"core_node19",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:33543/solr",
[junit4] 2> "node_name":"127.0.0.1:33543_solr",
[junit4] 2> "numShards":"5",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard5",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "force_set_state":"false",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 2727015 DEBUG (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.Overseer processMessage: queueSize: 7, message = {
[junit4] 2> "core":"testMaxOps_collection_shard3_replica_n8",
[junit4] 2> "core_node_name":"core_node11",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:33543/solr",
[junit4] 2> "node_name":"127.0.0.1:33543_solr",
[junit4] 2> "numShards":"5",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "force_set_state":"false",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 2727017 DEBUG (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.Overseer processMessage: queueSize: 10, message = {
[junit4] 2> "core":"testMaxOps_collection_shard2_replica_n6",
[junit4] 2> "core_node_name":"core_node9",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:33543/solr",
[junit4] 2> "node_name":"127.0.0.1:33543_solr",
[junit4] 2> "numShards":"5",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "force_set_state":"false",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 2727017 DEBUG (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.Overseer processMessage: queueSize: 10, message = {
[junit4] 2> "core":"testMaxOps_collection_shard4_replica_n12",
[junit4] 2> "core_node_name":"core_node15",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:43052/solr",
[junit4] 2> "node_name":"127.0.0.1:43052_solr",
[junit4] 2> "numShards":"5",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard4",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "force_set_state":"false",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 2727017 DEBUG (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.Overseer processMessage: queueSize: 10, message = {
[junit4] 2> "core":"testMaxOps_collection_shard4_replica_n14",
[junit4] 2> "core_node_name":"core_node17",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:33543/solr",
[junit4] 2> "node_name":"127.0.0.1:33543_solr",
[junit4] 2> "numShards":"5",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard4",
[junit4] 2> "collection":"testMaxOps_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "force_set_state":"false",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 2727271 DEBUG (ScheduledTrigger-13588-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2728016 INFO (qtp2122523878-114155) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard2 r:core_node7 x:testMaxOps_collection_shard2_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2728034 INFO (qtp2122523878-114158) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard1 r:core_node5 x:testMaxOps_collection_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2728035 INFO (qtp2122523878-114159) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard3 r:core_node13 x:testMaxOps_collection_shard3_replica_n10] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2728070 INFO (qtp2122523878-114155) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard2 r:core_node7 x:testMaxOps_collection_shard2_replica_n4] o.a.s.s.IndexSchema [testMaxOps_collection_shard2_replica_n4] Schema name=minimal
[junit4] 2> 2728070 INFO (qtp628727391-114167) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard5 r:core_node19 x:testMaxOps_collection_shard5_replica_n16] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2728075 INFO (qtp628727391-114163) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard4 r:core_node17 x:testMaxOps_collection_shard4_replica_n14] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2728076 INFO (qtp2122523878-114157) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard5 r:core_node20 x:testMaxOps_collection_shard5_replica_n18] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2728077 INFO (qtp2122523878-114155) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard2 r:core_node7 x:testMaxOps_collection_shard2_replica_n4] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2728077 INFO (qtp2122523878-114155) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard2 r:core_node7 x:testMaxOps_collection_shard2_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'testMaxOps_collection_shard2_replica_n4' using configuration from collection testMaxOps_collection, trusted=true
[junit4] 2> 2728078 INFO (qtp628727391-114166) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard3 r:core_node11 x:testMaxOps_collection_shard3_replica_n8] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2728078 INFO (qtp2122523878-114155) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard2 r:core_node7 x:testMaxOps_collection_shard2_replica_n4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43052.solr.core.testMaxOps_collection.shard2.replica_n4' (registry 'solr.core.testMaxOps_collection.shard2.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c80d78a
[junit4] 2> 2728079 INFO (qtp2122523878-114155) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard2 r:core_node7 x:testMaxOps_collection_shard2_replica_n4] o.a.s.c.SolrCore [[testMaxOps_collection_shard2_replica_n4] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E3777760836268B3-001/tempDir-001/node1/testMaxOps_collection_shard2_replica_n4], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E3777760836268B3-001/tempDir-001/node1/./testMaxOps_collection_shard2_replica_n4/data/]
[junit4] 2> 2728083 INFO (qtp2122523878-114158) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard1 r:core_node5 x:testMaxOps_collection_shard1_replica_n2] o.a.s.s.IndexSchema [testMaxOps_collection_shard1_replica_n2] Schema name=minimal
[junit4] 2> 2728087 INFO (qtp628727391-114163) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard4 r:core_node17 x:testMaxOps_collection_shard4_replica_n14] o.a.s.s.IndexSchema [testMaxOps_collection_shard4_replica_n14] Schema name=minimal
[junit4] 2> 2728089 INFO (qtp628727391-114169) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard1 r:core_node3 x:testMaxOps_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2728090 INFO (qtp2122523878-114161) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard4 r:core_node15 x:testMaxOps_collection_shard4_replica_n12] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2728090 INFO (qtp628727391-114163) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard4 r:core_node17 x:testMaxOps_collection_shard4_replica_n14] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2728091 INFO (qtp628727391-114163) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard4 r:core_node17 x:testMaxOps_collection_shard4_replica_n14] o.a.s.c.CoreContainer Creating SolrCore 'testMaxOps_collection_shard4_replica_n14' using configuration from collection testMaxOps_collection, trusted=true
[junit4] 2> 2728091 INFO (qtp628727391-114163) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard4 r:core_node17 x:testMaxOps_collection_shard4_replica_n14] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33543.solr.core.testMaxOps_collection.shard4.replica_n14' (registry 'solr.core.testMaxOps_collection.shard4.replica_n14') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c80d78a
[junit4] 2> 2728092 INFO (qtp628727391-114163) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard4 r:core_node17 x:testMaxOps_collection_shard4_replica_n14] o.a.s.c.SolrCore [[testMaxOps_collection_shard4_replica_n14] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E3777760836268B3-001/tempDir-001/node2/testMaxOps_collection_shard4_replica_n14], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E3777760836268B3-001/tempDir-001/node2/./testMaxOps_collection_shard4_replica_n14/data/]
[junit4] 2> 2728092 INFO (qtp2122523878-114159) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard3 r:core_node13 x:testMaxOps_collection_shard3_replica_n10] o.a.s.s.IndexSchema [testMaxOps_collection_shard3_replica_n10] Schema name=minimal
[junit4] 2> 2728119 INFO (qtp628727391-114165) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard2 r:core_node9 x:testMaxOps_collection_shard2_replica_n6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2728125 INFO (qtp2122523878-114159) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard3 r:core_node13 x:testMaxOps_collection_shard3_replica_n10] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2728125 INFO (qtp2122523878-114159) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard3 r:core_node13 x:testMaxOps_collection_shard3_replica_n10] o.a.s.c.CoreContainer Creating SolrCore 'testMaxOps_collection_shard3_replica_n10' using configuration from collection testMaxOps_collection, trusted=true
[junit4] 2> 2728125 INFO (qtp2122523878-114158) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard1 r:core_node5 x:testMaxOps_collection_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2728125 INFO (qtp2122523878-114158) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard1 r:core_node5 x:testMaxOps_collection_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'testMaxOps_collection_shard1_replica_n2' using configuration from collection testMaxOps_collection, trusted=true
[junit4] 2> 2728126 INFO (qtp2122523878-114159) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard3 r:core_node13 x:testMaxOps_collection_shard3_replica_n10] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43052.solr.core.testMaxOps_collection.shard3.replica_n10' (registry 'solr.core.testMaxOps_collection.shard3.replica_n10') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c80d78a
[junit4] 2> 2728126 INFO (qtp2122523878-114159) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard3 r:core_node13 x:testMaxOps_collection_shard3_replica_n10] o.a.s.c.SolrCore [[testMaxOps_collection_shard3_replica_n10] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E3777760836268B3-001/tempDir-001/node1/testMaxOps_collection_shard3_replica_n10], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E3777760836268B3-001/tempDir-001/node1/./testMaxOps_collection_shard3_replica_n10/data/]
[junit4] 2> 2728127 INFO (qtp2122523878-114158) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard1 r:core_node5 x:testMaxOps_collection_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43052.solr.core.testMaxOps_collection.shard1.replica_n2' (registry 'solr.core.testMaxOps_collection.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c80d78a
[junit4] 2> 2728127 INFO (qtp2122523878-114158) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard1 r:core_node5 x:testMaxOps_collection_shard1_replica_n2] o.a.s.c.SolrCore [[testMaxOps_collection_shard1_replica_n2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E3777760836268B3-001/tempDir-001/node1/testMaxOps_collection_shard1_replica_n2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E3777760836268B3-001/tempDir-001/node1/./testMaxOps_collection_shard1_replica_n2/data/]
[junit4] 2> 2728128 INFO (qtp628727391-114169) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard1 r:core_node3 x:testMaxOps_collection_shard1_replica_n1] o.a.s.s.IndexSchema [testMaxOps_collection_shard1_replica_n1] Schema name=minimal
[junit4] 2> 2728128 INFO (qtp628727391-114167) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard5 r:core_node19 x:testMaxOps_collection_shard5_replica_n16] o.a.s.s.IndexSchema [testMaxOps_collection_shard5_replica_n16] Schema name=minimal
[junit4] 2> 2728132 INFO (qtp2122523878-114157) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard5 r:core_node20 x:testMaxOps_collection_shard5_replica_n18] o.a.s.s.IndexSchema [testMaxOps_collection_shard5_replica_n18] Schema name=minimal
[junit4] 2> 2728132 INFO (qtp628727391-114169) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard1 r:core_node3 x:testMaxOps_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2728132 INFO (qtp628727391-114169) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard1 r:core_node3 x:testMaxOps_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testMaxOps_collection_shard1_replica_n1' using configuration from collection testMaxOps_collection, trusted=true
[junit4] 2> 2728133 INFO (qtp628727391-114169) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard1 r:core_node3 x:testMaxOps_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33543.solr.core.testMaxOps_collection.shard1.replica_n1' (registry 'solr.core.testMaxOps_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c80d78a
[junit4] 2> 2728133 INFO (qtp628727391-114169) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard1 r:core_node3 x:testMaxOps_collection_shard1_replica_n1] o.a.s.c.SolrCore [[testMaxOps_collection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E3777760836268B3-001/tempDir-001/node2/testMaxOps_collection_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E3777760836268B3-001/tempDir-001/node2/./testMaxOps_collection_shard1_replica_n1/data/]
[junit4] 2> 2728134 INFO (qtp628727391-114167) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard5 r:core_node19 x:testMaxOps_collection_shard5_replica_n16] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2728134 INFO (qtp628727391-114167) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard5 r:core_node19 x:testMaxOps_collection_shard5_replica_n16] o.a.s.c.CoreContainer Creating SolrCore 'testMaxOps_collection_shard5_replica_n16' using configuration from collection testMaxOps_collection, trusted=true
[junit4] 2> 2728138 INFO (qtp2122523878-114161) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard4 r:core_node15 x:testMaxOps_collection_shard4_replica_n12] o.a.s.s.IndexSchema [testMaxOps_collection_shard4_replica_n12] Schema name=minimal
[junit4] 2> 2728138 INFO (qtp2122523878-114157) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard5 r:core_node20 x:testMaxOps_collection_shard5_replica_n18] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2728138 INFO (qtp2122523878-114157) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard5 r:core_node20 x:testMaxOps_collection_shard5_replica_n18] o.a.s.c.CoreContainer Creating SolrCore 'testMaxOps_collection_shard5_replica_n18' using configuration from collection testMaxOps_collection, trusted=true
[junit4] 2> 2728143 INFO (qtp628727391-114167) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard5 r:core_node19 x:testMaxOps_collection_shard5_replica_n16] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33543.solr.core.testMaxOps_collection.shard5.replica_n16' (registry 'solr.core.testMaxOps_collection.shard5.replica_n16') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c80d78a
[junit4] 2> 2728143 INFO (qtp628727391-114167) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard5 r:core_node19 x:testMaxOps_collection_shard5_replica_n16] o.a.s.c.SolrCore [[testMaxOps_collection_shard5_replica_n16] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E3777760836268B3-001/tempDir-001/node2/testMaxOps_collection_shard5_replica_n16], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E3777760836268B3-001/tempDir-001/node2/./testMaxOps_collection_shard5_replica_n16/data/]
[junit4] 2> 2728145 INFO (qtp2122523878-114157) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard5 r:core_node20 x:testMaxOps_collection_shard5_replica_n18] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43052.solr.core.testMaxOps_collection.shard5.replica_n18' (registry 'solr.core.testMaxOps_collection.shard5.replica_n18') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c80d78a
[junit4] 2> 2728145 INFO (qtp2122523878-114161) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard4 r:core_node15 x:testMaxOps_collection_shard4_replica_n12] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2728145 INFO (qtp2122523878-114161) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard4 r:core_node15 x:testMaxOps_collection_shard4_replica_n12] o.a.s.c.CoreContainer Creating SolrCore 'testMaxOps_collection_shard4_replica_n12' using configuration from collection testMaxOps_collection, trusted=true
[junit4] 2> 2728145 INFO (qtp2122523878-114157) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard5 r:core_node20 x:testMaxOps_collection_shard5_replica_n18] o.a.s.c.SolrCore [[testMaxOps_collection_shard5_replica_n18] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E3777760836268B3-001/tempDir-001/node1/testMaxOps_collection_shard5_replica_n18], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E3777760836268B3-001/tempDir-001/node1/./testMaxOps_collection_shard5_replica_n18/data/]
[junit4] 2> 2728146 INFO (qtp2122523878-114161) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard4 r:core_node15 x:testMaxOps_collection_shard4_replica_n12] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43052.solr.core.testMaxOps_collection.shard4.replica_n12' (registry 'solr.core.testMaxOps_collection.shard4.replica_n12') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c80d78a
[junit4] 2> 2728146 INFO (qtp2122523878-114161) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard4 r:core_node15 x:testMaxOps_collection_shard4_replica_n12] o.a.s.c.SolrCore [[testMaxOps_collection_shard4_replica_n12] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E3777760836268B3-001/tempDir-001/node1/testMaxOps_collection_shard4_replica_n12], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E3777760836268B3-001/tempDir-001/node1/./testMaxOps_collection_shard4_replica_n12/data/]
[junit4] 2> 2728148 INFO (qtp628727391-114165) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard2 r:core_node9 x:testMaxOps_collection_shard2_replica_n6] o.a.s.s.IndexSchema [testMaxOps_collection_shard2_replica_n6] Schema name=minimal
[junit4] 2> 2728149 INFO (qtp628727391-114166) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard3 r:core_node11 x:testMaxOps_collection_shard3_replica_n8] o.a.s.s.IndexSchema [testMaxOps_collection_shard3_replica_n8] Schema name=minimal
[junit4] 2> 2728174 INFO (qtp628727391-114165) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard2 r:core_node9 x:testMaxOps_collection_shard2_replica_n6] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2728174 INFO (qtp628727391-114165) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard2 r:core_node9 x:testMaxOps_collection_shard2_replica_n6] o.a.s.c.CoreContainer Creating SolrCore 'testMaxOps_collection_shard2_replica_n6' using configuration from collection testMaxOps_collection, trusted=true
[junit4] 2> 2728175 INFO (qtp628727391-114165) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard2 r:core_node9 x:testMaxOps_collection_shard2_replica_n6] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33543.solr.core.testMaxOps_collection.shard2.replica_n6' (registry 'solr.core.testMaxOps_collection.shard2.replica_n6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c80d78a
[junit4] 2> 2728175 INFO (qtp628727391-114165) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard2 r:core_node9 x:testMaxOps_collection_shard2_replica_n6] o.a.s.c.SolrCore [[testMaxOps_collection_shard2_replica_n6] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E3777760836268B3-001/tempDir-001/node2/testMaxOps_collection_shard2_replica_n6], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E3777760836268B3-001/tempDir-001/node2/./testMaxOps_collection_shard2_replica_n6/data/]
[junit4] 2> 2728179 INFO (qtp628727391-114166) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard3 r:core_node11 x:testMaxOps_collection_shard3_replica_n8] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2728179 INFO (qtp628727391-114166) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard3 r:core_node11 x:testMaxOps_collection_shard3_replica_n8] o.a.s.c.CoreContainer Creating SolrCore 'testMaxOps_collection_shard3_replica_n8' using configuration from collection testMaxOps_collection, trusted=true
[junit4] 2> 2728179 INFO (qtp628727391-114166) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard3 r:core_node11 x:testMaxOps_collection_shard3_replica_n8] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33543.solr.core.testMaxOps_collection.shard3.replica_n8' (registry 'solr.core.testMaxOps_collection.shard3.replica_n8') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c80d78a
[junit4] 2> 2728179 INFO (qtp628727391-114166) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard3 r:core_node11 x:testMaxOps_collection_shard3_replica_n8] o.a.s.c.SolrCore [[testMaxOps_collection_shard3_replica_n8] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E3777760836268B3-001/tempDir-001/node2/testMaxOps_collection_shard3_replica_n8], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_E3777760836268B3-001/tempDir-001/node2/./testMaxOps_collection_shard3_replica_n8/data/]
[junit4] 2> 2728273 DEBUG (ScheduledTrigger-13588-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2728419 INFO (qtp628727391-114163) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard4 r:core_node17 x:testMaxOps_collection_shard4_replica_n14] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2728420 INFO (qtp628727391-114163) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard4 r:core_node17 x:testMaxOps_collection_shard4_replica_n14] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2728427 INFO (qtp628727391-114163) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard4 r:core_node17 x:testMaxOps_collection_shard4_replica_n14] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2728427 INFO (qtp628727391-114163) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard4 r:core_node17 x:testMaxOps_collection_shard4_replica_n14] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2728444 INFO (qtp628727391-114163) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard4 r:core_node17 x:testMaxOps_collection_shard4_replica_n14] o.a.s.s.SolrIndexSearcher Opening [Searcher@663bda43[testMaxOps_collection_shard4_replica_n14] main]
[junit4] 2> 2728446 INFO (qtp628727391-114163) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard4 r:core_node17 x:testMaxOps_collection_shard4_replica_n14] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2728447 INFO (qtp628727391-114163) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard4 r:core_node17 x:testMaxOps_collection_shard4_replica_n14] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2728454 INFO (qtp628727391-114163) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard4 r:core_node17 x:testMaxOps_collection_shard4_replica_n14] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2728467 INFO (searcherExecutor-13603-thread-1-processing-n:127.0.0.1:33543_solr x:testMaxOps_collection_shard4_replica_n14 c:testMaxOps_collection s:shard4 r:core_node17) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard4 r:core_node17 x:testMaxOps_collection_shard4_replica_n14] o.a.s.c.SolrCore [testMaxOps_collection_shard4_replica_n14] Registered new searcher Searcher@663bda43[testMaxOps_collection_shard4_replica_n14] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2728467 INFO (qtp628727391-114163) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard4 r:core_node17 x:testMaxOps_collection_shard4_replica_n14] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1617815252049592320
[junit4] 2> 2728476 INFO (qtp628727391-114163) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard4 r:core_node17 x:testMaxOps_collection_shard4_replica_n14] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testMaxOps_collection/terms/shard4 to Terms{values={core_node17=0}, version=0}
[junit4] 2> 2728483 DEBUG (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard4",
[junit4] 2> "collection":"testMaxOps_collection"} current state version: 0
[junit4] 2> 2728484 INFO (qtp628727391-114163) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard4 r:core_node17 x:testMaxOps_collection_shard4_replica_n14] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard4: total=2 found=1 timeoutin=9998ms
[junit4] 2> 2728502 INFO (qtp628727391-114169) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard1 r:core_node3 x:testMaxOps_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2728502 INFO (qtp628727391-114169) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard1 r:core_node3 x:testMaxOps_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2728507 INFO (qtp2122523878-114155) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard2 r:core_node7 x:testMaxOps_collection_shard2_replica_n4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2728507 INFO (qtp2122523878-114155) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard2 r:core_node7 x:testMaxOps_collection_shard2_replica_n4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2728509 INFO (qtp628727391-114169) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard1 r:core_node3 x:testMaxOps_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2728509 INFO (qtp628727391-114169) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard1 r:core_node3 x:testMaxOps_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2728514 INFO (qtp2122523878-114155) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard2 r:core_node7 x:testMaxOps_collection_shard2_replica_n4] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2728514 INFO (qtp2122523878-114155) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard2 r:core_node7 x:testMaxOps_collection_shard2_replica_n4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2728516 INFO (qtp2122523878-114157) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard5 r:core_node20 x:testMaxOps_collection_shard5_replica_n18] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2728516 INFO (qtp2122523878-114157) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard5 r:core_node20 x:testMaxOps_collection_shard5_replica_n18] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2728522 INFO (qtp628727391-114166) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard3 r:core_node11 x:testMaxOps_collection_shard3_replica_n8] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2728522 INFO (qtp628727391-114166) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard3 r:core_node11 x:testMaxOps_collection_shard3_replica_n8] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2728523 INFO (qtp2122523878-114157) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard5 r:core_node20 x:testMaxOps_collection_shard5_replica_n18] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2728523 INFO (qtp628727391-114167) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard5 r:core_node19 x:testMaxOps_collection_shard5_replica_n16] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2728523 INFO (qtp628727391-114167) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard5 r:core_node19 x:testMaxOps_collection_shard5_replica_n16] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2728523 INFO (qtp2122523878-114157) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard5 r:core_node20 x:testMaxOps_collection_shard5_replica_n18] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2728528 INFO (qtp628727391-114169) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard1 r:core_node3 x:testMaxOps_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3a1c2b51[testMaxOps_collection_shard1_replica_n1] main]
[junit4] 2> 2728528 INFO (qtp2122523878-114155) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard2 r:core_node7 x:testMaxOps_collection_shard2_replica_n4] o.a.s.s.SolrIndexSearcher Opening [Searcher@7843bc48[testMaxOps_collection_shard2_replica_n4] main]
[junit4] 2> 2728528 INFO (qtp628727391-114166) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard3 r:core_node11 x:testMaxOps_collection_shard3_replica_n8] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2728528 INFO (qtp628727391-114166) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard3 r:core_node11 x:testMaxOps_collection_shard3_replica_n8] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2728530 INFO (qtp628727391-114169) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard1 r:core_node3 x:testMaxOps_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2728530 INFO (qtp628727391-114167) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard5 r:core_node19 x:testMaxOps_collection_shard5_replica_n16] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2728530 INFO (qtp628727391-114167) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard5 r:core_node19 x:testMaxOps_collection_shard5_replica_n16] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2728530 INFO (qtp2122523878-114155) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard2 r:core_node7 x:testMaxOps_collection_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2728530 INFO (qtp628727391-114169) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard1 r:core_node3 x:testMaxOps_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2728531 INFO (qtp2122523878-114155) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard2 r:core_node7 x:testMaxOps_collection_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2728536 INFO (qtp2122523878-114155) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard2 r:core_node7 x:testMaxOps_collection_shard2_replica_n4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2728537 INFO (qtp2122523878-114161) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard4 r:core_node15 x:testMaxOps_collection_shard4_replica_n12] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2728537 INFO (qtp2122523878-114161) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard4 r:core_node15 x:testMaxOps_collection_shard4_replica_n12] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2728537 INFO (qtp628727391-114169) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard1 r:core_node3 x:testMaxOps_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2728537 INFO (qtp628727391-114169) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard1 r:core_node3 x:testMaxOps_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1617815252122992640
[junit4] 2> 2728538 INFO (qtp2122523878-114157) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard5 r:core_node20 x:testMaxOps_collection_shard5_replica_n18] o.a.s.s.SolrIndexSearcher Opening [Searcher@4a227fe9[testMaxOps_collection_shard5_replica_n18] main]
[junit4] 2> 2728540 INFO (qtp2122523878-114158) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard1 r:core_node5 x:testMaxOps_collection_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2728540 INFO (qtp2122523878-114158) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard1 r:core_node5 x:testMaxOps_collection_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2728541 INFO (qtp2122523878-114157) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard5 r:core_node20 x:testMaxOps_collection_shard5_replica_n18] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2728542 INFO (qtp2122523878-114157) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard5 r:core_node20 x:testMaxOps_collection_shard5_replica_n18] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2728545 INFO (qtp628727391-114166) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard3 r:core_node11 x:testMaxOps_collection_shard3_replica_n8] o.a.s.s.SolrIndexSearcher Opening [Searcher@3f6dff3c[testMaxOps_collection_shard3_replica_n8] main]
[junit4] 2> 2728546 INFO (qtp628727391-114166) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard3 r:core_node11 x:testMaxOps_collection_shard3_replica_n8] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2728546 INFO (qtp628727391-114167) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard5 r:core_node19 x:testMaxOps_collection_shard5_replica_n16] o.a.s.s.SolrIndexSearcher Opening [Searcher@e77c4d7[testMaxOps_collection_shard5_replica_n16] main]
[junit4] 2> 2728547 INFO (qtp628727391-114169) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard1 r:core_node3 x:testMaxOps_collection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testMaxOps_collection/terms/shard1 to Terms{values={core_node3=0}, version=0}
[junit4] 2> 2728547 INFO (qtp628727391-114166) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard3 r:core_node11 x:testMaxOps_collection_shard3_replica_n8] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2728548 INFO (qtp628727391-114167) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard5 r:core_node19 x:testMaxOps_collection_shard5_replica_n16] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2728548 INFO (searcherExecutor-13602-thread-1-processing-n:127.0.0.1:43052_solr x:testMaxOps_collection_shard2_replica_n4 c:testMaxOps_collection s:shard2 r:core_node7) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard2 r:core_node7 x:testMaxOps_collection_shard2_replica_n4] o.a.s.c.SolrCore [testMaxOps_collection_shard2_replica_n4] Registered new searcher Searcher@7843bc48[testMaxOps_collection_shard2_replica_n4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2728548 INFO (qtp2122523878-114155) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard2 r:core_node7 x:testMaxOps_collection_shard2_replica_n4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1617815252134526976
[junit4] 2> 2728548 INFO (qtp2122523878-114161) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard4 r:core_node15 x:testMaxOps_collection_shard4_replica_n12] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2728548 INFO (qtp2122523878-114158) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard1 r:core_node5 x:testMaxOps_collection_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2728548 INFO (qtp2122523878-114161) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard4 r:core_node15 x:testMaxOps_collection_shard4_replica_n12] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2728548 INFO (qtp2122523878-114158) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard1 r:core_node5 x:testMaxOps_collection_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2728549 INFO (searcherExecutor-13606-thread-1-processing-n:127.0.0.1:33543_solr x:testMaxOps_collection_shard1_replica_n1 c:testMaxOps_collection s:shard1 r:core_node3) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard1 r:core_node3 x:testMaxOps_collection_shard1_replica_n1] o.a.s.c.SolrCore [testMaxOps_collection_shard1_replica_n1] Registered new searcher Searcher@3a1c2b51[testMaxOps_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2728550 INFO (qtp2122523878-114157) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard5 r:core_node20 x:testMaxOps_collection_shard5_replica_n18] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2728550 INFO (qtp2122523878-114157) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard5 r:core_node20 x:testMaxOps_collection_shard5_replica_n18] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1617815252136624128
[junit4] 2> 2728550 INFO (qtp628727391-114167) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard5 r:core_node19 x:testMaxOps_collection_shard5_replica_n16] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2728551 INFO (qtp2122523878-114159) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard3 r:core_node13 x:testMaxOps_collection_shard3_replica_n10] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2728551 INFO (qtp628727391-114165) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard2 r:core_node9 x:testMaxOps_collection_shard2_replica_n6] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2728551 INFO (qtp2122523878-114159) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard3 r:core_node13 x:testMaxOps_collection_shard3_replica_n10] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2728551 INFO (qtp628727391-114165) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard2 r:core_node9 x:testMaxOps_collection_shard2_replica_n6] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2728552 INFO (qtp628727391-114166) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard3 r:core_node11 x:testMaxOps_collection_shard3_replica_n8] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2728552 INFO (qtp628727391-114166) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard3 r:core_node11 x:testMaxOps_collection_shard3_replica_n8] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1617815252138721280
[junit4] 2> 2728556 INFO (qtp628727391-114167) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard5 r:core_node19 x:testMaxOps_collection_shard5_replica_n16] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2728556 INFO (qtp628727391-114167) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard5 r:core_node19 x:testMaxOps_collection_shard5_replica_n16] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1617815252142915584
[junit4] 2> 2728560 INFO (qtp628727391-114165) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard2 r:core_node9 x:testMaxOps_collection_shard2_replica_n6] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2728560 INFO (qtp628727391-114165) [n:127.0.0.1:33543_solr c:testMaxOps_collection s:shard2 r:core_node9 x:testMaxOps_collection_shard2_replica_n6] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2728561 INFO (qtp2122523878-114159) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard3 r:core_node13 x:testMaxOps_collection_shard3_replica_n10] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2728561 INFO (qtp2122523878-114159) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard3 r:core_node13 x:testMaxOps_collection_shard3_replica_n10] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2728562 INFO (searcherExecutor-13608-thread-1-processing-n:127.0.0.1:43052_solr x:testMaxOps_collection_shard5_replica_n18 c:testMaxOps_collection s:shard5 r:core_node20) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard5 r:core_node20 x:testMaxOps_collection_shard5_replica_n18] o.a.s.c.SolrCore [testMaxOps_collection_shard5_replica_n18] Registered new searcher Searcher@4a227fe9[testMaxOps_collection_shard5_replica_n18] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2728562 INFO (qtp2122523878-114157) [n:127.0.0.1:43052_solr c:testMaxOps_collection s:shard5 r:core_node20 x:testMaxOps_collection_shard5_replica_n18] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testMaxOps_collection/terms/shard5 to Terms{values={core_node20=0}, version=0}
[junit4] 2> 2728563 DEBUG (OverseerStateUpdate-72319976252375044-127.0.0.1:43052_solr-n_0000000000) [n:127.0.0.1:43052_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testMaxOps_collection"} current state version: 0
[...truncated too long message...]
ve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/null581450962
[copy] Copying 240 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/null581450962
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/null581450962
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 50ms :: artifacts dl 5ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 287 minutes 39 seconds
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
at hudson.FilePath$34.hasMatch(FilePath.java:2678)
at hudson.FilePath$34.invoke(FilePath.java:2557)
at hudson.FilePath$34.invoke(FilePath.java:2547)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:955)
at hudson.FilePath.act(FilePath.java:1036)
at hudson.FilePath.act(FilePath.java:1025)
at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1819)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused: hudson.FilePath$TunneledInterruptedException
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2920)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
at hudson.FilePath.act(FilePath.java:1038)
at hudson.FilePath.act(FilePath.java:1025)
at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1819)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1709 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1709/
2 tests failed.
FAILED: org.apache.lucene.document.TestLatLonLineShapeQueries.testRandomBig
Error Message:
Java heap space
Stack Trace:
java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([EA127355C5DCC27D:6D450EDA5485BEFD]:0)
at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:84)
at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:57)
at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:168)
at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:154)
at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:141)
at org.apache.lucene.util.bkd.OfflinePointWriter.append(OfflinePointWriter.java:75)
at org.apache.lucene.util.bkd.BKDWriter.add(BKDWriter.java:284)
at org.apache.lucene.codecs.lucene60.Lucene60PointsWriter$1.visit(Lucene60PointsWriter.java:120)
at org.apache.lucene.codecs.PointsWriter$1$1$1.visit(PointsWriter.java:117)
at org.apache.lucene.index.AssertingLeafReader$AssertingIntersectVisitor.visit(AssertingLeafReader.java:1162)
at org.apache.lucene.util.bkd.BKDReader.visitCompressedDocValues(BKDReader.java:522)
at org.apache.lucene.util.bkd.BKDReader.visitDocValues(BKDReader.java:485)
at org.apache.lucene.util.bkd.BKDReader.intersect(BKDReader.java:577)
at org.apache.lucene.util.bkd.BKDReader.intersect(BKDReader.java:610)
at org.apache.lucene.util.bkd.BKDReader.intersect(BKDReader.java:600)
at org.apache.lucene.util.bkd.BKDReader.intersect(BKDReader.java:600)
at org.apache.lucene.util.bkd.BKDReader.intersect(BKDReader.java:610)
at org.apache.lucene.util.bkd.BKDReader.intersect(BKDReader.java:600)
at org.apache.lucene.util.bkd.BKDReader.intersect(BKDReader.java:610)
at org.apache.lucene.util.bkd.BKDReader.intersect(BKDReader.java:361)
at org.apache.lucene.index.AssertingLeafReader$AssertingPointValues.intersect(AssertingLeafReader.java:1051)
at org.apache.lucene.codecs.PointsWriter$1$1.intersect(PointsWriter.java:105)
at org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.writeField(Lucene60PointsWriter.java:113)
at org.apache.lucene.codecs.PointsWriter.mergeOneField(PointsWriter.java:62)
at org.apache.lucene.codecs.PointsWriter.merge(PointsWriter.java:191)
at org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.merge(Lucene60PointsWriter.java:145)
at org.apache.lucene.codecs.asserting.AssertingPointsFormat$AssertingPointsWriter.merge(AssertingPointsFormat.java:142)
at org.apache.lucene.index.SegmentMerger.mergePoints(SegmentMerger.java:201)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:161)
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4483)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4078)
at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
FAILED: org.apache.solr.client.solrj.io.stream.MathExpressionTest.testGammaDistribution
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([4D135FCF35EACE53:7069746116926444]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.client.solrj.io.stream.MathExpressionTest.testGammaDistribution(MathExpressionTest.java:4372)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 10293 lines...]
[junit4] Suite: org.apache.lucene.document.TestLatLonLineShapeQueries
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestLatLonLineShapeQueries -Dtests.method=testRandomBig -Dtests.seed=EA127355C5DCC27D -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=es-CL -Dtests.timezone=Asia/Amman -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 134s J2 | TestLatLonLineShapeQueries.testRandomBig <<<
[junit4] > Throwable #1: java.lang.OutOfMemoryError: Java heap space
[junit4] > at __randomizedtesting.SeedInfo.seed([EA127355C5DCC27D:6D450EDA5485BEFD]:0)
[junit4] > at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:84)
[junit4] > at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:57)
[junit4] > at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:168)
[junit4] > at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:154)
[junit4] > at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:141)
[junit4] > at org.apache.lucene.util.bkd.OfflinePointWriter.append(OfflinePointWriter.java:75)
[junit4] > at org.apache.lucene.util.bkd.BKDWriter.add(BKDWriter.java:284)
[junit4] > at org.apache.lucene.codecs.lucene60.Lucene60PointsWriter$1.visit(Lucene60PointsWriter.java:120)
[junit4] > at org.apache.lucene.codecs.PointsWriter$1$1$1.visit(PointsWriter.java:117)
[junit4] > at org.apache.lucene.index.AssertingLeafReader$AssertingIntersectVisitor.visit(AssertingLeafReader.java:1162)
[junit4] > at org.apache.lucene.util.bkd.BKDReader.visitCompressedDocValues(BKDReader.java:522)
[junit4] > at org.apache.lucene.util.bkd.BKDReader.visitDocValues(BKDReader.java:485)
[junit4] > at org.apache.lucene.util.bkd.BKDReader.intersect(BKDReader.java:577)
[junit4] > at org.apache.lucene.util.bkd.BKDReader.intersect(BKDReader.java:610)
[junit4] > at org.apache.lucene.util.bkd.BKDReader.intersect(BKDReader.java:600)
[junit4] > at org.apache.lucene.util.bkd.BKDReader.intersect(BKDReader.java:600)
[junit4] > at org.apache.lucene.util.bkd.BKDReader.intersect(BKDReader.java:610)
[junit4] > at org.apache.lucene.util.bkd.BKDReader.intersect(BKDReader.java:600)
[junit4] > at org.apache.lucene.util.bkd.BKDReader.intersect(BKDReader.java:610)
[junit4] > at org.apache.lucene.util.bkd.BKDReader.intersect(BKDReader.java:361)
[junit4] > at org.apache.lucene.index.AssertingLeafReader$AssertingPointValues.intersect(AssertingLeafReader.java:1051)
[junit4] > at org.apache.lucene.codecs.PointsWriter$1$1.intersect(PointsWriter.java:105)
[junit4] > at org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.writeField(Lucene60PointsWriter.java:113)
[junit4] > at org.apache.lucene.codecs.PointsWriter.mergeOneField(PointsWriter.java:62)
[junit4] > at org.apache.lucene.codecs.PointsWriter.merge(PointsWriter.java:191)
[junit4] > at org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.merge(Lucene60PointsWriter.java:145)
[junit4] > at org.apache.lucene.codecs.asserting.AssertingPointsFormat$AssertingPointsWriter.merge(AssertingPointsFormat.java:142)
[junit4] > at org.apache.lucene.index.SegmentMerger.mergePoints(SegmentMerger.java:201)
[junit4] > at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:161)
[junit4] > at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4483)
[junit4] > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4078)
[junit4] > at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/build/sandbox/test/J2/temp/lucene.document.TestLatLonLineShapeQueries_EA127355C5DCC27D-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene80), sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@355e0eb9), locale=es-CL, timezone=Asia/Amman
[junit4] 2> NOTE: Linux 4.4.0-137-generic amd64/Oracle Corporation 1.8.0_191 (64-bit)/cpus=4,threads=1,free=225922936,total=506986496
[junit4] 2> NOTE: All tests run in this JVM: [TestTermAutomatonQuery, TestLatLonShape, TestDoubleRangeField, TestFieldCacheTermsFilter, TestBigIntegerPoint, TestLatLonLineShapeQueries]
[junit4] Completed [15/23 (1!)] on J2 in 566.93s, 4 tests, 1 error <<< FAILURES!
[...truncated 21 lines...]
[junit4] JVM J2: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/build/sandbox/test/temp/junit4-J2-20181129_111000_11255662920745060007.sysout
[junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: Java heap space
[junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/heapdumps/java_pid22668.hprof ...
[junit4] Heap dump file created [445488417 bytes in 0.729 secs]
[junit4] <<< JVM J2: EOF ----
[...truncated 5698 lines...]
[junit4] JVM J0: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/temp/junit4-J0-20181129_122947_9207225920377233311757.sysout
[junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
[junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/heapdumps/java_pid4559.hprof ...
[junit4] Heap dump file created [574090837 bytes in 3.301 secs]
[junit4] <<< JVM J0: EOF ----
[...truncated 852 lines...]
[junit4] Suite: org.apache.solr.client.solrj.io.stream.MathExpressionTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.io.stream.MathExpressionTest_4D135FCF35EACE53-001/init-core-data-001
[junit4] 1> 13:44:25.801 [Thread-203] ERROR org.apache.zookeeper.server.ZooKeeperServer - ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 1> 13:44:26.008 [jetty-launcher-419-thread-3] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 1> 13:44:26.029 [jetty-launcher-419-thread-2] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 1> 13:44:26.042 [jetty-launcher-419-thread-1] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 1> 13:44:26.078 [jetty-launcher-419-thread-4] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] IGNOR/A 0.00s J1 | MathExpressionTest.testDistributions
[junit4] > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] IGNOR/A 0.00s J1 | MathExpressionTest.testMultiVariateNormalDistribution
[junit4] > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=MathExpressionTest -Dtests.method=testGammaDistribution -Dtests.seed=4D135FCF35EACE53 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=ar -Dtests.timezone=America/Argentina/Jujuy -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 0.46s J1 | MathExpressionTest.testGammaDistribution <<<
[junit4] > Throwable #1: java.lang.AssertionError
[junit4] > at __randomizedtesting.SeedInfo.seed([4D135FCF35EACE53:7069746116926444]:0)
[junit4] > at org.apache.solr.client.solrj.io.stream.MathExpressionTest.testGammaDistribution(MathExpressionTest.java:4372)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 1> 13:44:48.860 [SUITE-MathExpressionTest-seed#[4D135FCF35EACE53]-worker] ERROR org.apache.zookeeper.server.ZooKeeperServer - ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.io.stream.MathExpressionTest_4D135FCF35EACE53-001
[junit4] 2> Nov 29, 2018 1:44:48 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene80): {multiDefault=PostingsFormat(name=Asserting), price_f=PostingsFormat(name=Asserting), intDefault=PostingsFormat(name=LuceneVarGapDocFreqInterval), test_dt=PostingsFormat(name=LuceneVarGapFixedInterval), price_i=Lucene50(blocksize=128), test_t=Lucene50(blocksize=128), col_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), order_i=PostingsFormat(name=Asserting), loc_p_1_d1=PostingsFormat(name=LuceneVarGapFixedInterval), _version_=PostingsFormat(name=LuceneVarGapDocFreqInterval), loc_p_0_d1=PostingsFormat(name=LuceneVarGapDocFreqInterval), id=Lucene50(blocksize=128), text=PostingsFormat(name=LuceneVarGapFixedInterval), miles_i=PostingsFormat(name=LuceneVarGapDocFreqInterval), timestamp=PostingsFormat(name=Asserting)}, docValues:{col_s=DocValuesFormat(name=Direct), order_i=DocValuesFormat(name=Lucene70), _version_=DocValuesFormat(name=Direct), multiDefault=DocValuesFormat(name=Lucene70), price_f=DocValuesFormat(name=Lucene70), intDefault=DocValuesFormat(name=Direct), test_dt=DocValuesFormat(name=Lucene70), id=DocValuesFormat(name=Asserting), price_i=DocValuesFormat(name=Asserting), miles_i=DocValuesFormat(name=Direct), timestamp=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=237, maxMBSortInHeap=6.608805654019506, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@75290392), locale=ar, timezone=America/Argentina/Jujuy
[junit4] 2> NOTE: Linux 4.4.0-137-generic amd64/Oracle Corporation 1.8.0_191 (64-bit)/cpus=4,threads=1,free=116946712,total=389021696
[junit4] 2> NOTE: All tests run in this JVM: [GetByIdTest, ModifiableSolrParamsTest, JettyWebappTest, TestTimeSource, HyperbolicCosineEvaluatorTest, NormalDistributionEvaluatorTest, HttpSolrClientBuilderTest, TestDocumentObjectBinder, AppendEvaluatorTest, QueryResponseTest, EqualToEvaluatorTest, TestJavaBinCodec, GraphExpressionTest, ConcurrentUpdateSolrClientBadInputTest, TestUpdateRequestCodec, ClientUtilsTest, CeilingEvaluatorTest, TestEmbeddedSolrServer, LengthEvaluatorTest, ArcTangentEvaluatorTest, ConcatOperationTest, TestZkConfigManager, TestFastInputStream, CumulativeProbabilityEvaluatorTest, UsingSolrJRefGuideExamplesTest, TermsFacetMapTest, SolrExampleJettyTest, TestPathTrie, SolrExceptionTest, SolrQueryTest, SolrSchemalessExampleTest, LargeVolumeEmbeddedTest, LargeVolumeJettyTest, MergeIndexesEmbeddedTest, BasicHttpSolrClientTest, CloudSolrClientBuilderTest, CloudSolrClientCacheTest, ConcurrentUpdateSolrClientMultiCollectionTest, HttpClientUtilTest, HttpSolrClientSSLAuthConPoolTest, LBHttpSolrClientBadInputTest, LBHttpSolrClientBuilderTest, TestCloudSolrClientConnections, JDBCStreamTest, MathExpressionTest]
[junit4] Completed [109/174 (1!)] on J1 in 23.71s, 114 tests, 1 failure, 2 skipped <<< FAILURES!
[...truncated 8159 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:651: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:585: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid22668.hprof
* java_pid4559.hprof
Total time: 289 minutes 43 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1708 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1708/
5 tests failed.
FAILED: org.apache.solr.cloud.RestartWhileUpdatingTest.test
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([813A3D1CE93389BC]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.RestartWhileUpdatingTest
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([813A3D1CE93389BC]:0)
FAILED: org.apache.solr.cloud.TestRebalanceLeaders.test
Error Message:
Error from server at https://127.0.0.1:36779/_l/qc: KeeperErrorCode = Session expired for /configs/conf1
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:36779/_l/qc: KeeperErrorCode = Session expired for /configs/conf1
at __randomizedtesting.SeedInfo.seed([813A3D1CE93389BC:96E02C647CFE444]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1260)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1676)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1703)
at org.apache.solr.cloud.TestRebalanceLeaders.test(TestRebalanceLeaders.java:71)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1010)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration
Error Message:
did not finish processing in time
Stack Trace:
java.lang.AssertionError: did not finish processing in time
at __randomizedtesting.SeedInfo.seed([813A3D1CE93389BC:B8B4845CC6CC4042]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration(IndexSizeTriggerTest.java:320)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.handler.component.SearchHandlerTest.testRequireZkConnectedDistrib
Error Message:
Error from server at http://127.0.0.1:42559/solr: KeeperErrorCode = Session expired for /configs/testRequireZkConnectedDistribCollectionConfig
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:42559/solr: KeeperErrorCode = Session expired for /configs/testRequireZkConnectedDistribCollectionConfig
at __randomizedtesting.SeedInfo.seed([813A3D1CE93389BC:39B4049D2234C21B]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.handler.component.SearchHandlerTest.testRequireZkConnectedDistrib(SearchHandlerTest.java:224)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13426 lines...]
[junit4] Suite: org.apache.solr.handler.component.SearchHandlerTest
[junit4] 2> 13089 INFO (SUITE-SearchHandlerTest-seed#[813A3D1CE93389BC]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.handler.component.SearchHandlerTest_813A3D1CE93389BC-001/init-core-data-001
[junit4] 2> 13094 WARN (SUITE-SearchHandlerTest-seed#[813A3D1CE93389BC]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=21 numCloses=21
[junit4] 2> 13101 INFO (SUITE-SearchHandlerTest-seed#[813A3D1CE93389BC]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 13103 INFO (SUITE-SearchHandlerTest-seed#[813A3D1CE93389BC]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 13103 INFO (SUITE-SearchHandlerTest-seed#[813A3D1CE93389BC]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore
[junit4] 2> 13104 INFO (SUITE-SearchHandlerTest-seed#[813A3D1CE93389BC]-worker) [ ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 13238 INFO (SUITE-SearchHandlerTest-seed#[813A3D1CE93389BC]-worker) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 13327 INFO (SUITE-SearchHandlerTest-seed#[813A3D1CE93389BC]-worker) [ ] o.a.s.s.IndexSchema [null] Schema name=test
[junit4] 2> 13545 WARN (SUITE-SearchHandlerTest-seed#[813A3D1CE93389BC]-worker) [ ] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.WordDelimiterFilterFactory]. Please consult documentation how to replace it accordingly.
[junit4] 2> 13721 INFO (SUITE-SearchHandlerTest-seed#[813A3D1CE93389BC]-worker) [ ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 14035 INFO (SUITE-SearchHandlerTest-seed#[813A3D1CE93389BC]-worker) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 14035 INFO (SUITE-SearchHandlerTest-seed#[813A3D1CE93389BC]-worker) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 14063 INFO (SUITE-SearchHandlerTest-seed#[813A3D1CE93389BC]-worker) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d0b869e
[junit4] 2> 14063 INFO (SUITE-SearchHandlerTest-seed#[813A3D1CE93389BC]-worker) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d0b869e
[junit4] 2> 14063 INFO (SUITE-SearchHandlerTest-seed#[813A3D1CE93389BC]-worker) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d0b869e
[junit4] 2> 14067 INFO (coreLoadExecutor-25-thread-1) [ x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 14184 INFO (coreLoadExecutor-25-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 14258 INFO (coreLoadExecutor-25-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 14527 INFO (coreLoadExecutor-25-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 14566 INFO (coreLoadExecutor-25-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1, trusted=true
[junit4] 2> 14582 INFO (coreLoadExecutor-25-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d0b869e
[junit4] 2> 14582 INFO (coreLoadExecutor-25-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.handler.component.SearchHandlerTest_813A3D1CE93389BC-001/init-core-data-001/]
[junit4] 2> 14586 INFO (coreLoadExecutor-25-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=9, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.3515625, noCFSRatio=0.0]
[junit4] 2> 14837 INFO (coreLoadExecutor-25-thread-1) [ x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 14837 INFO (coreLoadExecutor-25-thread-1) [ x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 14839 INFO (coreLoadExecutor-25-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 14839 INFO (coreLoadExecutor-25-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 14841 INFO (coreLoadExecutor-25-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=48, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=86.53125, floorSegmentMB=0.2783203125, forceMergeDeletesPctAllowed=10.108578546459245, segmentsPerTier=49.0, maxCFSSegmentSizeMB=1.2734375, noCFSRatio=1.0, deletesPctAllowed=21.506662956809635
[junit4] 2> 14842 INFO (coreLoadExecutor-25-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2b210a2[collection1] main]
[junit4] 2> 14844 INFO (coreLoadExecutor-25-thread-1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
[junit4] 2> 15012 INFO (coreLoadExecutor-25-thread-1) [ x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 15297 INFO (coreLoadExecutor-25-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 15299 INFO (searcherExecutor-26-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
[junit4] 2> 15314 INFO (searcherExecutor-26-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
[junit4] 2> 15314 INFO (searcherExecutor-26-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
[junit4] 2> 15314 INFO (searcherExecutor-26-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
[junit4] 2> 15315 INFO (searcherExecutor-26-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
[junit4] 2> 15315 INFO (searcherExecutor-26-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
[junit4] 2> 15316 INFO (searcherExecutor-26-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
[junit4] 2> 15318 INFO (searcherExecutor-26-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
[junit4] 2> 15318 INFO (searcherExecutor-26-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
[junit4] 2> 15321 INFO (searcherExecutor-26-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2b210a2[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 15322 INFO (coreLoadExecutor-25-thread-1) [ x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1618353479730855936
[junit4] 2> 15331 INFO (SUITE-SearchHandlerTest-seed#[813A3D1CE93389BC]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore end
[junit4] 2> 15339 INFO (TEST-SearchHandlerTest.testRequireZkConnected-seed#[813A3D1CE93389BC]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testRequireZkConnected
[junit4] 2> 15349 INFO (TEST-SearchHandlerTest.testRequireZkConnected-seed#[813A3D1CE93389BC]) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 5 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.handler.component.SearchHandlerTest_813A3D1CE93389BC-001/tempDir-001
[junit4] 2> 15372 INFO (TEST-SearchHandlerTest.testRequireZkConnected-seed#[813A3D1CE93389BC]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 15389 INFO (Thread-3) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 15390 INFO (Thread-3) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 15483 INFO (TEST-SearchHandlerTest.testRequireZkConnected-seed#[813A3D1CE93389BC]) [ ] o.a.s.c.ZkTestServer start zk server on port:42711
[junit4] 2> 15566 ERROR (Thread-3) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 15636 INFO (zkConnectionManagerCallback-8-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 15683 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x105fd012ea90000, likely client has closed socket
[junit4] 2> 16048 INFO (jetty-launcher-5-thread-5) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_191-b12
[junit4] 2> 16055 INFO (jetty-launcher-5-thread-1) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_191-b12
[junit4] 2> 16055 INFO (jetty-launcher-5-thread-2) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_191-b12
[junit4] 2> 16055 INFO (jetty-launcher-5-thread-3) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_191-b12
[junit4] 2> 16055 INFO (jetty-launcher-5-thread-4) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_191-b12
[junit4] 2> 16159 INFO (jetty-launcher-5-thread-5) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 16159 INFO (jetty-launcher-5-thread-5) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 16163 INFO (jetty-launcher-5-thread-5) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 16219 INFO (jetty-launcher-5-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 16219 INFO (jetty-launcher-5-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 16219 INFO (jetty-launcher-5-thread-1) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 16229 INFO (jetty-launcher-5-thread-5) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@24db72f2{/solr,null,AVAILABLE}
[junit4] 2> 16231 INFO (jetty-launcher-5-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@334e1649{/solr,null,AVAILABLE}
[junit4] 2> 16278 INFO (jetty-launcher-5-thread-4) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 16278 INFO (jetty-launcher-5-thread-4) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 16278 INFO (jetty-launcher-5-thread-4) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 16303 INFO (jetty-launcher-5-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5262ec2{/solr,null,AVAILABLE}
[junit4] 2> 16305 INFO (jetty-launcher-5-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 16305 INFO (jetty-launcher-5-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 16305 INFO (jetty-launcher-5-thread-3) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 16305 INFO (jetty-launcher-5-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@71a36f09{/solr,null,AVAILABLE}
[junit4] 2> 16306 INFO (jetty-launcher-5-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@605abd3c{HTTP/1.1,[http/1.1]}{127.0.0.1:41999}
[junit4] 2> 16306 INFO (jetty-launcher-5-thread-1) [ ] o.e.j.s.Server Started @16344ms
[junit4] 2> 16306 INFO (jetty-launcher-5-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41999}
[junit4] 2> 16307 INFO (jetty-launcher-5-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7177c619{HTTP/1.1,[http/1.1]}{127.0.0.1:33086}
[junit4] 2> 16308 INFO (jetty-launcher-5-thread-4) [ ] o.e.j.s.Server Started @16346ms
[junit4] 2> 16308 INFO (jetty-launcher-5-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33086}
[junit4] 2> 16339 INFO (jetty-launcher-5-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 16339 INFO (jetty-launcher-5-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 16339 INFO (jetty-launcher-5-thread-2) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 16340 INFO (jetty-launcher-5-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5d19da44{/solr,null,AVAILABLE}
[junit4] 2> 16345 INFO (jetty-launcher-5-thread-5) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4b17e401{HTTP/1.1,[http/1.1]}{127.0.0.1:35797}
[junit4] 2> 16345 INFO (jetty-launcher-5-thread-5) [ ] o.e.j.s.Server Started @16383ms
[junit4] 2> 16345 INFO (jetty-launcher-5-thread-5) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35797}
[junit4] 2> 16346 INFO (jetty-launcher-5-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2de097d2{HTTP/1.1,[http/1.1]}{127.0.0.1:34271}
[junit4] 2> 16346 INFO (jetty-launcher-5-thread-2) [ ] o.e.j.s.Server Started @16384ms
[junit4] 2> 16346 INFO (jetty-launcher-5-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34271}
[junit4] 2> 16346 INFO (jetty-launcher-5-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@a658495{HTTP/1.1,[http/1.1]}{127.0.0.1:46780}
[junit4] 2> 16347 INFO (jetty-launcher-5-thread-3) [ ] o.e.j.s.Server Started @16385ms
[junit4] 2> 16347 INFO (jetty-launcher-5-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46780}
[junit4] 2> 16366 INFO (jetty-launcher-5-thread-3) [ ] o.a.s.u.c.SSLCredentialProviderFactory Processing SSL Credential Provider chain: env;sysprop
[junit4] 2> 16370 ERROR (jetty-launcher-5-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 16370 INFO (jetty-launcher-5-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 16370 INFO (jetty-launcher-5-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 16370 INFO (jetty-launcher-5-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 16370 INFO (jetty-launcher-5-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 16371 INFO (jetty-launcher-5-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-11-28T05:15:49.535Z
[junit4] 2> 16372 ERROR (jetty-launcher-5-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 16371 ERROR (jetty-launcher-5-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 16374 INFO (jetty-launcher-5-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 16374 INFO (jetty-launcher-5-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 16374 INFO (jetty-launcher-5-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 16374 INFO (jetty-launcher-5-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 16371 ERROR (jetty-launcher-5-thread-5) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 16371 ERROR (jetty-launcher-5-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 16375 INFO (jetty-launcher-5-thread-5) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 16375 INFO (jetty-launcher-5-thread-4) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 16375 INFO (jetty-launcher-5-thread-5) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 16375 INFO (jetty-launcher-5-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 16375 INFO (jetty-launcher-5-thread-5) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 16375 INFO (jetty-launcher-5-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 16375 INFO (jetty-launcher-5-thread-5) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 16375 INFO (jetty-launcher-5-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 16375 INFO (jetty-launcher-5-thread-5) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-11-28T05:15:49.539Z
[junit4] 2> 16375 INFO (jetty-launcher-5-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-11-28T05:15:49.539Z
[junit4] 2> 16375 INFO (jetty-launcher-5-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-11-28T05:15:49.538Z
[junit4] 2> 16375 INFO (jetty-launcher-5-thread-3) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 16391 INFO (jetty-launcher-5-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 16391 INFO (jetty-launcher-5-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 16391 INFO (jetty-launcher-5-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 16391 INFO (jetty-launcher-5-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-11-28T05:15:49.555Z
[junit4] 2> 16413 INFO (zkConnectionManagerCallback-10-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 16432 INFO (jetty-launcher-5-thread-5) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 16434 INFO (zkConnectionManagerCallback-16-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 16435 INFO (zkConnectionManagerCallback-14-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 16436 INFO (zkConnectionManagerCallback-12-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 16439 INFO (zkConnectionManagerCallback-18-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 16440 INFO (jetty-launcher-5-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 16440 INFO (jetty-launcher-5-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 16451 INFO (jetty-launcher-5-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 16459 INFO (jetty-launcher-5-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 16927 INFO (jetty-launcher-5-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42711/solr
[junit4] 2> 16955 INFO (jetty-launcher-5-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42711/solr
[junit4] 2> 16979 INFO (jetty-launcher-5-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42711/solr
[junit4] 2> 17014 INFO (zkConnectionManagerCallback-31-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 17020 INFO (jetty-launcher-5-thread-5) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42711/solr
[junit4] 2> 17055 INFO (zkConnectionManagerCallback-28-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 17055 INFO (zkConnectionManagerCallback-25-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 17078 INFO (zkConnectionManagerCallback-34-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 17126 INFO (zkConnectionManagerCallback-36-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 17131 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x105fd012ea90008, likely client has closed socket
[junit4] 2> 17135 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x105fd012ea90009, likely client has closed socket
[junit4] 2> 17144 INFO (zkConnectionManagerCallback-40-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 17186 INFO (zkConnectionManagerCallback-38-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 17199 INFO (zkConnectionManagerCallback-42-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 17222 INFO (jetty-launcher-5-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42711/solr
[junit4] 2> 17302 INFO (zkConnectionManagerCallback-46-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 17329 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x105fd012ea9000e, likely client has closed socket
[junit4] 2> 17355 INFO (zkConnectionManagerCallback-56-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 17579 INFO (jetty-launcher-5-thread-3) [n:127.0.0.1:46780_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46780_solr
[junit4] 2> 17584 INFO (jetty-launcher-5-thread-3) [n:127.0.0.1:46780_solr ] o.a.s.c.Overseer Overseer (id=73743150441103375-127.0.0.1:46780_solr-n_0000000000) starting
[junit4] 2> 17847 INFO (zkConnectionManagerCallback-63-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 17857 INFO (jetty-launcher-5-thread-3) [n:127.0.0.1:46780_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42711/solr ready
[junit4] 2> 17924 INFO (jetty-launcher-5-thread-5) [n:127.0.0.1:35797_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35797_solr
[junit4] 2> 17958 INFO (jetty-launcher-5-thread-1) [n:127.0.0.1:41999_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41999_solr
[junit4] 2> 17958 INFO (jetty-launcher-5-thread-4) [n:127.0.0.1:33086_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33086_solr
[junit4] 2> 17962 INFO (jetty-launcher-5-thread-2) [n:127.0.0.1:34271_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34271_solr
[junit4] 2> 18011 INFO (zkCallback-37-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 18011 INFO (zkCallback-35-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 18012 INFO (zkCallback-41-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 18012 INFO (zkCallback-39-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 18012 INFO (zkCallback-62-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 18017 INFO (zkCallback-55-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 18115 INFO (zkConnectionManagerCallback-68-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 18136 INFO (jetty-launcher-5-thread-5) [n:127.0.0.1:35797_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 18139 INFO (jetty-launcher-5-thread-5) [n:127.0.0.1:35797_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42711/solr ready
[junit4] 2> 18144 INFO (zkConnectionManagerCallback-73-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 18157 INFO (jetty-launcher-5-thread-5) [n:127.0.0.1:35797_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 18235 INFO (jetty-launcher-5-thread-2) [n:127.0.0.1:34271_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 18236 INFO (jetty-launcher-5-thread-2) [n:127.0.0.1:34271_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42711/solr ready
[junit4] 2> 18278 INFO (OverseerStateUpdate-73743150441103375-127.0.0.1:46780_solr-n_0000000000) [n:127.0.0.1:46780_solr ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:46780_solr
[junit4] 2> 18287 INFO (jetty-launcher-5-thread-2) [n:127.0.0.1:34271_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 18291 INFO (jetty-launcher-5-thread-3) [n:127.0.0.1:46780_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 18291 INFO (jetty-launcher-5-thread-3) [n:127.0.0.1:46780_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46780_solr
[junit4] 2> 18294 INFO (zkConnectionManagerCallback-79-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 18311 INFO (zkCallback-55-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 18311 INFO (zkCallback-37-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 18311 INFO (zkCallback-62-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 18311 INFO (zkCallback-41-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 18311 INFO (zkCallback-39-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 18312 INFO (zkCallback-35-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 18328 INFO (jetty-launcher-5-thread-5) [n:127.0.0.1:35797_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35797.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d0b869e
[junit4] 2> 18329 INFO (jetty-launcher-5-thread-5) [n:127.0.0.1:35797_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35797.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d0b869e
[junit4] 2> 18329 INFO (jetty-launcher-5-thread-5) [n:127.0.0.1:35797_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35797.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d0b869e
[junit4] 2> 18340 INFO (zkCallback-72-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 18340 INFO (zkConnectionManagerCallback-84-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 18340 INFO (zkCallback-67-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 18349 INFO (jetty-launcher-5-thread-5) [n:127.0.0.1:35797_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.handler.component.SearchHandlerTest_813A3D1CE93389BC-001/tempDir-001/node5/.
[junit4] 2> 18417 INFO (jetty-launcher-5-thread-3) [n:127.0.0.1:46780_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 18448 INFO (jetty-launcher-5-thread-1) [n:127.0.0.1:41999_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 18449 INFO (jetty-launcher-5-thread-4) [n:127.0.0.1:33086_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 18450 INFO (jetty-launcher-5-thread-1) [n:127.0.0.1:41999_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42711/solr ready
[junit4] 2> 18451 INFO (jetty-launcher-5-thread-4) [n:127.0.0.1:33086_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42711/solr ready
[junit4] 2> 18462 INFO (jetty-launcher-5-thread-2) [n:127.0.0.1:34271_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34271.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d0b869e
[junit4] 2> 18462 INFO (jetty-launcher-5-thread-2) [n:127.0.0.1:34271_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34271.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d0b869e
[junit4] 2> 18462 INFO (jetty-launcher-5-thread-2) [n:127.0.0.1:34271_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34271.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d0b869e
[junit4] 2> 18464 INFO (jetty-launcher-5-thread-2) [n:127.0.0.1:34271_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.handler.component.SearchHandlerTest_813A3D1CE93389BC-001/tempDir-001/node2/.
[junit4] 2> 18472 INFO (jetty-launcher-5-thread-3) [n:127.0.0.1:46780_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46780.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d0b869e
[junit4] 2> 18492 INFO (jetty-launcher-5-thread-3) [n:127.0.0.1:46780_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46780.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d0b869e
[junit4] 2> 18493 INFO (jetty-launcher-5-thread-3) [n:127.0.0.1:46780_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46780.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d0b869e
[junit4] 2> 18495 INFO (jetty-launcher-5-thread-3) [n:127.0.0.1:46780_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.handler.component.SearchHandlerTest_813A3D1CE93389BC-001/tempDir-001/node3/.
[junit4] 2> 18501 INFO (jetty-launcher-5-thread-4) [n:127.0.0.1:33086_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 18555 INFO (jetty-launcher-5-thread-1) [n:127.0.0.1:41999_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 18613 INFO (jetty-launcher-5-thread-4) [n:127.0.0.1:33086_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33086.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d0b869e
[junit4] 2> 18642 INFO (jetty-launcher-5-thread-4) [n:127.0.0.1:33086_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33086.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d0b869e
[junit4] 2> 18642 INFO (jetty-launcher-5-thread-4) [n:127.0.0.1:33086_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33086.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d0b869e
[junit4] 2> 18643 INFO (jetty-launcher-5-thread-4) [n:127.0.0.1:33086_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.handler.component.SearchHandlerTest_813A3D1CE93389BC-001/tempDir-001/node4/.
[junit4] 2> 18656 INFO (jetty-launcher-5-thread-1) [n:127.0.0.1:41999_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41999.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d0b869e
[junit4] 2> 18660 INFO (jetty-launcher-5-thread-1) [n:127.0.0.1:41999_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41999.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d0b869e
[junit4] 2> 18660 INFO (jetty-launcher-5-thread-1) [n:127.0.0.1:41999_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41999.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d0b869e
[junit4] 2> 18666 INFO (jetty-launcher-5-thread-1) [n:127.0.0.1:41999_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.handler.component.SearchHandlerTest_813A3D1CE93389BC-001/tempDir-001/node1/.
[junit4] 2> 19327 INFO (zkConnectionManagerCallback-90-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 19403 INFO (zkConnectionManagerCallback-93-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 20229 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x105fd012ea90016, likely client has closed socket
[junit4] 2> 20330 INFO (zkConnectionManagerCallback-97-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 20350 INFO (TEST-SearchHandlerTest.testRequireZkConnected-seed#[813A3D1CE93389BC]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 20353 INFO (TEST-SearchHandlerTest.testRequireZkConnected-seed#[813A3D1CE93389BC]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42711/solr ready
[junit4] 2> 20875 INFO (qtp858894873-58) [n:127.0.0.1:35797_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=testRequireZkConnectedCollectionConfig&name=testRequireZkConnectedCollection&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 20951 INFO (OverseerThreadFactory-67-thread-1-processing-n:127.0.0.1:46780_solr) [n:127.0.0.1:46780_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection testRequireZkConnectedCollection
[junit4] 2> 21268 INFO (qtp858894873-53) [n:127.0.0.1:35797_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=55
[junit4] 2> 21396 INFO (qtp320149958-87) [n:127.0.0.1:46780_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=1
[junit4] 2> 21415 INFO (qtp629161019-93) [n:127.0.0.1:34271_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=1
[junit4] 2> 21458 INFO (qtp757075289-69) [n:127.0.0.1:41999_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=1
[junit4] 2> 21476 INFO (qtp398979421-80) [n:127.0.0.1:33086_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=1
[junit4] 2> 21558 INFO (OverseerStateUpdate-73743150441103375-127.0.0.1:46780_solr-n_0000000000) [n:127.0.0.1:46780_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testRequireZkConnectedCollection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testRequireZkConnectedCollection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:33086/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 21584 INFO (OverseerStateUpdate-73743150441103375-127.0.0.1:46780_solr-n_0000000000) [n:127.0.0.1:46780_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testRequireZkConnectedCollection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testRequireZkConnectedCollection_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:41999/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 21593 INFO (OverseerStateUpdate-73743150441103375-127.0.0.1:46780_solr-n_0000000000) [n:127.0.0.1:46780_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testRequireZkConnectedCollection",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"testRequireZkConnectedCollection_shard2_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:34271/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 21603 INFO (OverseerStateUpdate-73743150441103375-127.0.0.1:46780_solr-n_0000000000) [n:127.0.0.1:46780_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testRequireZkConnectedCollection",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"testRequireZkConnectedCollection_shard2_replica_n6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:46780/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 21839 INFO (qtp757075289-52) [n:127.0.0.1:41999_solr x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=testRequireZkConnectedCollectionConfig&newCollection=true&name=testRequireZkConnectedCollection_shard1_replica_n2&action=CREATE&numShards=2&collection=testRequireZkConnectedCollection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 21840 INFO (qtp757075289-52) [n:127.0.0.1:41999_solr x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 21879 INFO (qtp629161019-64) [n:127.0.0.1:34271_solr x:testRequireZkConnectedCollection_shard2_replica_n4] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=testRequireZkConnectedCollectionConfig&newCollection=true&name=testRequireZkConnectedCollection_shard2_replica_n4&action=CREATE&numShards=2&collection=testRequireZkConnectedCollection&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 21880 INFO (qtp629161019-64) [n:127.0.0.1:34271_solr x:testRequireZkConnectedCollection_shard2_replica_n4] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 21901 INFO (qtp398979421-63) [n:127.0.0.1:33086_solr x:testRequireZkConnectedCollection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=testRequireZkConnectedCollectionConfig&newCollection=true&name=testRequireZkConnectedCollection_shard1_replica_n1&action=CREATE&numShards=2&collection=testRequireZkConnectedCollection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 21901 INFO (qtp398979421-63) [n:127.0.0.1:33086_solr x:testRequireZkConnectedCollection_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 21902 INFO (qtp320149958-62) [n:127.0.0.1:46780_solr x:testRequireZkConnectedCollection_shard2_replica_n6] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=testRequireZkConnectedCollectionConfig&newCollection=true&name=testRequireZkConnectedCollection_shard2_replica_n6&action=CREATE&numShards=2&collection=testRequireZkConnectedCollection&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 23009 INFO (qtp629161019-64) [n:127.0.0.1:34271_solr c:testRequireZkConnectedCollection s:shard2 r:core_node7 x:testRequireZkConnectedCollection_shard2_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 23018 INFO (qtp757075289-52) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 23040 INFO (qtp398979421-63) [n:127.0.0.1:33086_solr c:testRequireZkConnectedCollection s:shard1 r:core_node3 x:testRequireZkConnectedCollection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 23146 INFO (qtp398979421-63) [n:127.0.0.1:33086_solr c:testRequireZkConnectedCollection s:shard1 r:core_node3 x:testRequireZkConnectedCollection_shard1_replica_n1] o.a.s.s.IndexSchema [testRequireZkConnectedCollection_shard1_replica_n1] Schema name=test
[junit4] 2> 23176 INFO (qtp629161019-64) [n:127.0.0.1:34271_solr c:testRequireZkConnectedCollection s:shard2 r:core_node7 x:testRequireZkConnectedCollection_shard2_replica_n4] o.a.s.s.IndexSchema [testRequireZkConnectedCollection_shard2_replica_n4] Schema name=test
[junit4] 2> 23186 INFO (qtp757075289-52) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.s.IndexSchema [testRequireZkConnectedCollection_shard1_replica_n2] Schema name=test
[junit4] 2> 23291 INFO (qtp320149958-62) [n:127.0.0.1:46780_solr c:testRequireZkConnectedCollection s:shard2 r:core_node8 x:testRequireZkConnectedCollection_shard2_replica_n6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 23504 INFO (qtp320149958-62) [n:127.0.0.1:46780_solr c:testRequireZkConnectedCollection s:shard2 r:core_node8 x:testRequireZkConnectedCollection_shard2_replica_n6] o.a.s.s.IndexSchema [testRequireZkConnectedCollection_shard2_replica_n6] Schema name=test
[junit4] 2> 23845 INFO (qtp398979421-63) [n:127.0.0.1:33086_solr c:testRequireZkConnectedCollection s:shard1 r:core_node3 x:testRequireZkConnectedCollection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 23890 INFO (qtp629161019-64) [n:127.0.0.1:34271_solr c:testRequireZkConnectedCollection s:shard2 r:core_node7 x:testRequireZkConnectedCollection_shard2_replica_n4] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 23935 INFO (qtp757075289-52) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 23984 INFO (qtp398979421-63) [n:127.0.0.1:33086_solr c:testRequireZkConnectedCollection s:shard1 r:core_node3 x:testRequireZkConnectedCollection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testRequireZkConnectedCollection_shard1_replica_n1' using configuration from collection testRequireZkConnectedCollection, trusted=true
[junit4] 2> 24036 INFO (qtp629161019-64) [n:127.0.0.1:34271_solr c:testRequireZkConnectedCollection s:shard2 r:core_node7 x:testRequireZkConnectedCollection_shard2_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'testRequireZkConnectedCollection_shard2_replica_n4' using configuration from collection testRequireZkConnectedCollection, trusted=true
[junit4] 2> 24037 INFO (qtp320149958-62) [n:127.0.0.1:46780_solr c:testRequireZkConnectedCollection s:shard2 r:core_node8 x:testRequireZkConnectedCollection_shard2_replica_n6] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 24037 INFO (qtp398979421-63) [n:127.0.0.1:33086_solr c:testRequireZkConnectedCollection s:shard1 r:core_node3 x:testRequireZkConnectedCollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33086.solr.core.testRequireZkConnectedCollection.shard1.replica_n1' (registry 'solr.core.testRequireZkConnectedCollection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d0b869e
[junit4] 2> 24038 INFO (qtp398979421-63) [n:127.0.0.1:33086_solr c:testRequireZkConnectedCollection s:shard1 r:core_node3 x:testRequireZkConnectedCollection_shard1_replica_n1] o.a.s.c.SolrCore [[testRequireZkConnectedCollection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.handler.component.SearchHandlerTest_813A3D1CE93389BC-001/tempDir-001/node4/testRequireZkConnectedCollection_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.handler.component.SearchHandlerTest_813A3D1CE93389BC-001/tempDir-001/node4/./testRequireZkConnectedCollection_shard1_replica_n1/data/]
[junit4] 2> 24077 INFO (qtp629161019-64) [n:127.0.0.1:34271_solr c:testRequireZkConnectedCollection s:shard2 r:core_node7 x:testRequireZkConnectedCollection_shard2_replica_n4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34271.solr.core.testRequireZkConnectedCollection.shard2.replica_n4' (registry 'solr.core.testRequireZkConnectedCollection.shard2.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d0b869e
[junit4] 2> 24077 INFO (qtp629161019-64) [n:127.0.0.1:34271_solr c:testRequireZkConnectedCollection s:shard2 r:core_node7 x:testRequireZkConnectedCollection_shard2_replica_n4] o.a.s.c.SolrCore [[testRequireZkConnectedCollection_shard2_replica_n4] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.handler.component.SearchHandlerTest_813A3D1CE93389BC-001/tempDir-001/node2/testRequireZkConnectedCollection_shard2_replica_n4], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.handler.component.SearchHandlerTest_813A3D1CE93389BC-001/tempDir-001/node2/./testRequireZkConnectedCollection_shard2_replica_n4/data/]
[junit4] 2> 24090 INFO (qtp398979421-63) [n:127.0.0.1:33086_solr c:testRequireZkConnectedCollection s:shard1 r:core_node3 x:testRequireZkConnectedCollection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=9, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.3515625, noCFSRatio=0.0]
[junit4] 2> 24095 INFO (qtp757075289-52) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'testRequireZkConnectedCollection_shard1_replica_n2' using configuration from collection testRequireZkConnectedCollection, trusted=true
[junit4] 2> 24095 INFO (qtp757075289-52) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41999.solr.core.testRequireZkConnectedCollection.shard1.replica_n2' (registry 'solr.core.testRequireZkConnectedCollection.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d0b869e
[junit4] 2> 24096 INFO (qtp757075289-52) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.c.SolrCore [[testRequireZkConnectedCollection_shard1_replica_n2] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.handler.component.SearchHandlerTest_813A3D1CE93389BC-001/tempDir-001/node1/testRequireZkConnectedCollection_shard1_replica_n2], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.handler.component.SearchHandlerTest_813A3D1CE93389BC-001/tempDir-001/node1/./testRequireZkConnectedCollection_shard1_replica_n2/data/]
[junit4] 2> 24098 INFO (qtp629161019-64) [n:127.0.0.1:34271_solr c:testRequireZkConnectedCollection s:shard2 r:core_node7 x:testRequireZkConnectedCollection_shard2_replica_n4] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=9, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.3515625, noCFSRatio=0.0]
[junit4] 2> 24101 INFO (qtp757075289-52) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=9, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.3515625, noCFSRatio=0.0]
[junit4] 2> 24295 INFO (qtp320149958-62) [n:127.0.0.1:46780_solr c:testRequireZkConnectedCollection s:shard2 r:core_node8 x:testRequireZkConnectedCollection_shard2_replica_n6] o.a.s.c.CoreContainer Creating SolrCore 'testRequireZkConnectedCollection_shard2_replica_n6' using configuration from collection testRequireZkConnectedCollection, trusted=true
[junit4] 2> 24296 INFO (qtp320149958-62) [n:127.0.0.1:46780_solr c:testRequireZkConnectedCollection s:shard2 r:core_node8 x:testRequireZkConnectedCollection_shard2_replica_n6] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46780.solr.core.testRequireZkConnectedCollection.shard2.replica_n6' (registry 'solr.core.testRequireZkConnectedCollection.shard2.replica_n6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d0b869e
[junit4] 2> 24296 INFO (qtp320149958-62) [n:127.0.0.1:46780_solr c:testRequireZkConnectedCollection s:shard2 r:core_node8 x:testRequireZkConnectedCollection_shard2_replica_n6] o.a.s.c.SolrCore [[testRequireZkConnectedCollection_shard2_replica_n6] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.handler.component.SearchHandlerTest_813A3D1CE93389BC-001/tempDir-001/node3/testRequireZkConnectedCollection_shard2_replica_n6], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.handler.component.SearchHandlerTest_813A3D1CE93389BC-001/tempDir-001/node3/./testRequireZkConnectedCollection_shard2_replica_n6/data/]
[junit4] 2> 24310 INFO (qtp320149958-62) [n:127.0.0.1:46780_solr c:testRequireZkConnectedCollection s:shard2 r:core_node8 x:testRequireZkConnectedCollection_shard2_replica_n6] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=9, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.3515625, noCFSRatio=0.0]
[junit4] 2> 24767 INFO (qtp757075289-52) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 24767 INFO (qtp757075289-52) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 24769 INFO (qtp757075289-52) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 24769 INFO (qtp757075289-52) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 24770 INFO (qtp757075289-52) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=48, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=86.53125, floorSegmentMB=0.2783203125, forceMergeDeletesPctAllowed=10.108578546459245, segmentsPerTier=49.0, maxCFSSegmentSizeMB=1.2734375, noCFSRatio=1.0, deletesPctAllowed=21.506662956809635
[junit4] 2> 24771 INFO (qtp757075289-52) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@50adffc9[testRequireZkConnectedCollection_shard1_replica_n2] main]
[junit4] 2> 24772 INFO (qtp398979421-63) [n:127.0.0.1:33086_solr c:testRequireZkConnectedCollection s:shard1 r:core_node3 x:testRequireZkConnectedCollection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 24772 INFO (qtp398979421-63) [n:127.0.0.1:33086_solr c:testRequireZkConnectedCollection s:shard1 r:core_node3 x:testRequireZkConnectedCollection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 24773 INFO (qtp757075289-52) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
[junit4] 2> 24774 INFO (qtp398979421-63) [n:127.0.0.1:33086_solr c:testRequireZkConnectedCollection s:shard1 r:core_node3 x:testRequireZkConnectedCollection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 24774 INFO (qtp398979421-63) [n:127.0.0.1:33086_solr c:testRequireZkConnectedCollection s:shard1 r:core_node3 x:testRequireZkConnectedCollection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 24775 INFO (qtp398979421-63) [n:127.0.0.1:33086_solr c:testRequireZkConnectedCollection s:shard1 r:core_node3 x:testRequireZkConnectedCollection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=48, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=86.53125, floorSegmentMB=0.2783203125, forceMergeDeletesPctAllowed=10.108578546459245, segmentsPerTier=49.0, maxCFSSegmentSizeMB=1.2734375, noCFSRatio=1.0, deletesPctAllowed=21.506662956809635
[junit4] 2> 24775 INFO (qtp398979421-63) [n:127.0.0.1:33086_solr c:testRequireZkConnectedCollection s:shard1 r:core_node3 x:testRequireZkConnectedCollection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@267a8e6a[testRequireZkConnectedCollection_shard1_replica_n1] main]
[junit4] 2> 24804 INFO (qtp398979421-63) [n:127.0.0.1:33086_solr c:testRequireZkConnectedCollection s:shard1 r:core_node3 x:testRequireZkConnectedCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
[junit4] 2> 24823 INFO (qtp629161019-64) [n:127.0.0.1:34271_solr c:testRequireZkConnectedCollection s:shard2 r:core_node7 x:testRequireZkConnectedCollection_shard2_replica_n4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 24823 INFO (qtp629161019-64) [n:127.0.0.1:34271_solr c:testRequireZkConnectedCollection s:shard2 r:core_node7 x:testRequireZkConnectedCollection_shard2_replica_n4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 24824 INFO (qtp629161019-64) [n:127.0.0.1:34271_solr c:testRequireZkConnectedCollection s:shard2 r:core_node7 x:testRequireZkConnectedCollection_shard2_replica_n4] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 24824 INFO (qtp629161019-64) [n:127.0.0.1:34271_solr c:testRequireZkConnectedCollection s:shard2 r:core_node7 x:testRequireZkConnectedCollection_shard2_replica_n4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 24826 INFO (qtp629161019-64) [n:127.0.0.1:34271_solr c:testRequireZkConnectedCollection s:shard2 r:core_node7 x:testRequireZkConnectedCollection_shard2_replica_n4] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=48, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=86.53125, floorSegmentMB=0.2783203125, forceMergeDeletesPctAllowed=10.108578546459245, segmentsPerTier=49.0, maxCFSSegmentSizeMB=1.2734375, noCFSRatio=1.0, deletesPctAllowed=21.506662956809635
[junit4] 2> 24826 INFO (qtp629161019-64) [n:127.0.0.1:34271_solr c:testRequireZkConnectedCollection s:shard2 r:core_node7 x:testRequireZkConnectedCollection_shard2_replica_n4] o.a.s.s.SolrIndexSearcher Opening [Searcher@696706e2[testRequireZkConnectedCollection_shard2_replica_n4] main]
[junit4] 2> 24852 INFO (qtp629161019-64) [n:127.0.0.1:34271_solr c:testRequireZkConnectedCollection s:shard2 r:core_node7 x:testRequireZkConnectedCollection_shard2_replica_n4] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
[junit4] 2> 24872 INFO (qtp757075289-52) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 24873 INFO (qtp398979421-63) [n:127.0.0.1:33086_solr c:testRequireZkConnectedCollection s:shard1 r:core_node3 x:testRequireZkConnectedCollection_shard1_replica_n1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 24892 INFO (qtp320149958-62) [n:127.0.0.1:46780_solr c:testRequireZkConnectedCollection s:shard2 r:core_node8 x:testRequireZkConnectedCollection_shard2_replica_n6] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 24916 INFO (qtp320149958-62) [n:127.0.0.1:46780_solr c:testRequireZkConnectedCollection s:shard2 r:core_node8 x:testRequireZkConnectedCollection_shard2_replica_n6] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 24918 INFO (qtp320149958-62) [n:127.0.0.1:46780_solr c:testRequireZkConnectedCollection s:shard2 r:core_node8 x:testRequireZkConnectedCollection_shard2_replica_n6] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 24918 INFO (qtp320149958-62) [n:127.0.0.1:46780_solr c:testRequireZkConnectedCollection s:shard2 r:core_node8 x:testRequireZkConnectedCollection_shard2_replica_n6] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 24919 INFO (qtp320149958-62) [n:127.0.0.1:46780_solr c:testRequireZkConnectedCollection s:shard2 r:core_node8 x:testRequireZkConnectedCollection_shard2_replica_n6] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=48, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=86.53125, floorSegmentMB=0.2783203125, forceMergeDeletesPctAllowed=10.108578546459245, segmentsPerTier=49.0, maxCFSSegmentSizeMB=1.2734375, noCFSRatio=1.0, deletesPctAllowed=21.506662956809635
[junit4] 2> 24920 INFO (qtp320149958-62) [n:127.0.0.1:46780_solr c:testRequireZkConnectedCollection s:shard2 r:core_node8 x:testRequireZkConnectedCollection_shard2_replica_n6] o.a.s.s.SolrIndexSearcher Opening [Searcher@2ce0bee8[testRequireZkConnectedCollection_shard2_replica_n6] main]
[junit4] 2> 24966 INFO (qtp320149958-62) [n:127.0.0.1:46780_solr c:testRequireZkConnectedCollection s:shard2 r:core_node8 x:testRequireZkConnectedCollection_shard2_replica_n6] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
[junit4] 2> 24967 INFO (qtp629161019-64) [n:127.0.0.1:34271_solr c:testRequireZkConnectedCollection s:shard2 r:core_node7 x:testRequireZkConnectedCollection_shard2_replica_n4] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 25036 INFO (qtp320149958-62) [n:127.0.0.1:46780_solr c:testRequireZkConnectedCollection s:shard2 r:core_node8 x:testRequireZkConnectedCollection_shard2_replica_n6] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 25170 INFO (qtp757075289-52) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 25173 INFO (searcherExecutor-82-thread-1-processing-n:127.0.0.1:41999_solr x:testRequireZkConnectedCollection_shard1_replica_n2 c:testRequireZkConnectedCollection s:shard1 r:core_node5) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
[junit4] 2> 25174 INFO (searcherExecutor-82-thread-1-processing-n:127.0.0.1:41999_solr x:testRequireZkConnectedCollection_shard1_replica_n2 c:testRequireZkConnectedCollection s:shard1 r:core_node5) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
[junit4] 2> 25174 INFO (searcherExecutor-82-thread-1-processing-n:127.0.0.1:41999_solr x:testRequireZkConnectedCollection_shard1_replica_n2 c:testRequireZkConnectedCollection s:shard1 r:core_node5) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
[junit4] 2> 25174 INFO (searcherExecutor-82-thread-1-processing-n:127.0.0.1:41999_solr x:testRequireZkConnectedCollection_shard1_replica_n2 c:testRequireZkConnectedCollection s:shard1 r:core_node5) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
[junit4] 2> 25175 INFO (qtp757075289-52) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1618353490062475264
[junit4] 2> 25209 INFO (searcherExecutor-82-thread-1-processing-n:127.0.0.1:41999_solr x:testRequireZkConnectedCollection_shard1_replica_n2 c:testRequireZkConnectedCollection s:shard1 r:core_node5) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
[junit4] 2> 25210 INFO (searcherExecutor-82-thread-1-processing-n:127.0.0.1:41999_solr x:testRequireZkConnectedCollection_shard1_replica_n2 c:testRequireZkConnectedCollection s:shard1 r:core_node5) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
[junit4] 2> 25212 INFO (searcherExecutor-82-thread-1-processing-n:127.0.0.1:41999_solr x:testRequireZkConnectedCollection_shard1_replica_n2 c:testRequireZkConnectedCollection s:shard1 r:core_node5) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
[junit4] 2> 25253 INFO (searcherExecutor-82-thread-1-processing-n:127.0.0.1:41999_solr x:testRequireZkConnectedCollection_shard1_replica_n2 c:testRequireZkConnectedCollection s:shard1 r:core_node5) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
[junit4] 2> 25287 INFO (qtp398979421-63) [n:127.0.0.1:33086_solr c:testRequireZkConnectedCollection s:shard1 r:core_node3 x:testRequireZkConnectedCollection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 25287 INFO (qtp398979421-63) [n:127.0.0.1:33086_solr c:testRequireZkConnectedCollection s:shard1 r:core_node3 x:testRequireZkConnectedCollection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1618353490179915776
[junit4] 2> 25288 INFO (searcherExecutor-80-thread-1-processing-n:127.0.0.1:33086_solr x:testRequireZkConnectedCollection_shard1_replica_n1 c:testRequireZkConnectedCollection s:shard1 r:core_node3) [n:127.0.0.1:33086_solr c:testRequireZkConnectedCollection s:shard1 r:core_node3 x:testRequireZkConnectedCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
[junit4] 2> 25302 INFO (searcherExecutor-80-thread-1-processing-n:127.0.0.1:33086_solr x:testRequireZkConnectedCollection_shard1_replica_n1 c:testRequireZkConnectedCollection s:shard1 r:core_node3) [n:127.0.0.1:33086_solr c:testRequireZkConnectedCollection s:shard1 r:core_node3 x:testRequireZkConnectedCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
[junit4] 2> 25306 INFO (searcherExecutor-80-thread-1-processing-n:127.0.0.1:33086_solr x:testRequireZkConnectedCollection_shard1_replica_n1 c:testRequireZkConnectedCollection s:shard1 r:core_node3) [n:127.0.0.1:33086_solr c:testRequireZkConnectedCollection s:shard1 r:core_node3 x:testRequireZkConnectedCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
[junit4] 2> 25306 INFO (searcherExecutor-80-thread-1-processing-n:127.0.0.1:33086_solr x:testRequireZkConnectedCollection_shard1_replica_n1 c:testRequireZkConnectedCollection s:shard1 r:core_node3) [n:127.0.0.1:33086_solr c:testRequireZkConnectedCollection s:shard1 r:core_node3 x:testRequireZkConnectedCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
[junit4] 2> 25323 INFO (searcherExecutor-82-thread-1-processing-n:127.0.0.1:41999_solr x:testRequireZkConnectedCollection_shard1_replica_n2 c:testRequireZkConnectedCollection s:shard1 r:core_node5) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
[junit4] 2> 25362 INFO (searcherExecutor-80-thread-1-processing-n:127.0.0.1:33086_solr x:testRequireZkConnectedCollection_shard1_replica_n1 c:testRequireZkConnectedCollection s:shard1 r:core_node3) [n:127.0.0.1:33086_solr c:testRequireZkConnectedCollection s:shard1 r:core_node3 x:testRequireZkConnectedCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
[junit4] 2> 25368 INFO (qtp757075289-52) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/testRequireZkConnectedCollection/terms/shard1 to Terms{values={core_node5=0}, version=0}
[junit4] 2> 25372 INFO (searcherExecutor-82-thread-1-processing-n:127.0.0.1:41999_solr x:testRequireZkConnectedCollection_shard1_replica_n2 c:testRequireZkConnectedCollection s:shard1 r:core_node5) [n:127.0.0.1:41999_solr c:testRequireZkConnectedCollection s:shard1 r:core_node5 x:testRequireZkConnectedCollection_shard1_replica_n2] o.a.s.c.SolrCore [testRequireZkConnectedCollection_shard1_replica_n2] Registered new searcher Searcher@50adffc9[testRequireZkConnectedCollection_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 25391 INFO (searcherExecutor-80-thread-1-processing-n:127.0.0.1:33086_solr x:testRequireZkConnectedCollection_shard1_replica_n1 c:testRequireZkConnectedCollection s:shard1 r:core_node3) [n:127.0.0.1:33086_solr c:testRequireZkConnectedColl
[...truncated too long message...]
urce] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/null1041186694
[copy] Copying 240 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/null1041186694
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/null1041186694
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 171ms :: artifacts dl 17ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 622 minutes 43 seconds
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
at hudson.FilePath$34.hasMatch(FilePath.java:2678)
at hudson.FilePath$34.invoke(FilePath.java:2557)
at hudson.FilePath$34.invoke(FilePath.java:2547)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene2
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:955)
at hudson.FilePath.act(FilePath.java:1036)
at hudson.FilePath.act(FilePath.java:1025)
at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1819)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused: hudson.FilePath$TunneledInterruptedException
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2920)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
at hudson.FilePath.act(FilePath.java:1038)
at hudson.FilePath.act(FilePath.java:1025)
at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1819)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1707 - Still
unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1707/
3 tests failed.
FAILED: org.apache.solr.cloud.RestartWhileUpdatingTest.test
Error Message:
There are still nodes recoverying - waited for 320 seconds
Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 320 seconds
at __randomizedtesting.SeedInfo.seed([8D41AC50B277880A:515938A1C8BE5F2]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:185)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:920)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForThingsToLevelOut(AbstractFullDistribZkTestBase.java:1477)
at org.apache.solr.cloud.RestartWhileUpdatingTest.test(RestartWhileUpdatingTest.java:145)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1010)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.RestartWhileUpdatingTest
Error Message:
7 threads leaked from SUITE scope at org.apache.solr.cloud.RestartWhileUpdatingTest: 1) Thread[id=17990, name=searcherExecutor-6783-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=18201, name=searcherExecutor-6845-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=17953, name=searcherExecutor-6769-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=18033, name=searcherExecutor-6797-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=18166, name=searcherExecutor-6831-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=17910, name=searcherExecutor-6755-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=18129, name=searcherExecutor-6817-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 7 threads leaked from SUITE scope at org.apache.solr.cloud.RestartWhileUpdatingTest:
1) Thread[id=17990, name=searcherExecutor-6783-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=18201, name=searcherExecutor-6845-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=17953, name=searcherExecutor-6769-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=18033, name=searcherExecutor-6797-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=18166, name=searcherExecutor-6831-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=17910, name=searcherExecutor-6755-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
7) Thread[id=18129, name=searcherExecutor-6817-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([8D41AC50B277880A]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.RestartWhileUpdatingTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=17990, name=searcherExecutor-6783-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=18201, name=searcherExecutor-6845-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=17953, name=searcherExecutor-6769-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=18033, name=searcherExecutor-6797-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=18166, name=searcherExecutor-6831-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=17910, name=searcherExecutor-6755-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=18129, name=searcherExecutor-6817-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=17990, name=searcherExecutor-6783-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=18201, name=searcherExecutor-6845-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=17953, name=searcherExecutor-6769-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=18033, name=searcherExecutor-6797-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=18166, name=searcherExecutor-6831-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=17910, name=searcherExecutor-6755-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
7) Thread[id=18129, name=searcherExecutor-6817-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([8D41AC50B277880A]:0)
Build Log:
[...truncated 14861 lines...]
[junit4] Suite: org.apache.solr.cloud.RestartWhileUpdatingTest
[junit4] 2> 2569870 INFO (SUITE-RestartWhileUpdatingTest-seed#[8D41AC50B277880A]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_8D41AC50B277880A-001/init-core-data-001
[junit4] 2> 2569871 WARN (SUITE-RestartWhileUpdatingTest-seed#[8D41AC50B277880A]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
[junit4] 2> 2569871 INFO (SUITE-RestartWhileUpdatingTest-seed#[8D41AC50B277880A]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 2569873 INFO (SUITE-RestartWhileUpdatingTest-seed#[8D41AC50B277880A]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 2569873 INFO (SUITE-RestartWhileUpdatingTest-seed#[8D41AC50B277880A]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 2569876 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2569877 INFO (Thread-4335) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2569877 INFO (Thread-4335) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2569886 ERROR (Thread-4335) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2569977 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.ZkTestServer start zk server on port:45253
[junit4] 2> 2569979 INFO (zkConnectionManagerCallback-4838-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2569983 INFO (zkConnectionManagerCallback-4840-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2569991 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 2569993 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 2569995 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2569997 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 2569998 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 2570000 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 2570002 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 2570008 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 2570010 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 2570012 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 2570013 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 2570015 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 2570200 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_191-b12
[junit4] 2> 2570201 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2570201 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2570201 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 2570202 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2cffc7f6{/,null,AVAILABLE}
[junit4] 2> 2570204 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@28ec6d9e{HTTP/1.1,[http/1.1]}{127.0.0.1:38160}
[junit4] 2> 2570204 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.e.j.s.Server Started @2570266ms
[junit4] 2> 2570204 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_8D41AC50B277880A-001/tempDir-001/control/data, replicaType=NRT, hostContext=/, hostPort=38160, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_8D41AC50B277880A-001/control-001/cores}
[junit4] 2> 2570204 ERROR (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2570204 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2570204 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 2570204 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2570204 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2570204 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-11-27T02:34:29.484Z
[junit4] 2> 2570207 INFO (zkConnectionManagerCallback-4842-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2570208 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2570208 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_8D41AC50B277880A-001/control-001/solr.xml
[junit4] 2> 2570213 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2570213 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2570215 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@19988b7a, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2570676 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45253/solr
[junit4] 2> 2570677 INFO (zkConnectionManagerCallback-4846-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2570680 INFO (zkConnectionManagerCallback-4848-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2570799 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [n:127.0.0.1:38160_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:38160_
[junit4] 2> 2570801 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [n:127.0.0.1:38160_ ] o.a.s.c.Overseer Overseer (id=72347318965501956-127.0.0.1:38160_-n_0000000000) starting
[junit4] 2> 2570810 INFO (zkConnectionManagerCallback-4855-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2570812 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [n:127.0.0.1:38160_ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45253/solr ready
[junit4] 2> 2570813 INFO (OverseerStateUpdate-72347318965501956-127.0.0.1:38160_-n_0000000000) [n:127.0.0.1:38160_ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:38160_
[junit4] 2> 2570813 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [n:127.0.0.1:38160_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38160_
[junit4] 2> 2570819 INFO (OverseerStateUpdate-72347318965501956-127.0.0.1:38160_-n_0000000000) [n:127.0.0.1:38160_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2570828 INFO (zkCallback-4854-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2570845 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [n:127.0.0.1:38160_ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 2570872 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [n:127.0.0.1:38160_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19988b7a
[junit4] 2> 2570887 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [n:127.0.0.1:38160_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19988b7a
[junit4] 2> 2570887 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [n:127.0.0.1:38160_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19988b7a
[junit4] 2> 2570889 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [n:127.0.0.1:38160_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_8D41AC50B277880A-001/control-001/cores
[junit4] 2> 2570923 INFO (zkConnectionManagerCallback-4861-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2570924 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2570925 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45253/solr ready
[junit4] 2> 2570927 INFO (qtp1830451017-17870) [n:127.0.0.1:38160_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:38160_&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2570931 INFO (OverseerThreadFactory-6750-thread-1-processing-n:127.0.0.1:38160_) [n:127.0.0.1:38160_ ] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 2571038 INFO (qtp1830451017-17865) [n:127.0.0.1:38160_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2571044 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ x:control_collection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2571045 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ x:control_collection_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2572066 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2572082 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 2572199 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 2572220 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 2572220 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19988b7a
[junit4] 2> 2572221 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_8D41AC50B277880A-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_8D41AC50B277880A-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 2572225 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@39b4024c
[junit4] 2> 2572238 WARN (qtp1830451017-17866) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2572304 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2572304 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2572305 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommitted for 30000ms;
[junit4] 2> 2572306 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: if uncommitted for 3000ms;
[junit4] 2> 2572308 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=41, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 2572309 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4630caaa[control_collection_shard1_replica_n1] main]
[junit4] 2> 2572310 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2572310 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2572311 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2572311 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1618252735804604416
[junit4] 2> 2572314 INFO (searcherExecutor-6755-thread-1-processing-n:127.0.0.1:38160_ x:control_collection_shard1_replica_n1 c:control_collection s:shard1) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@4630caaa[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2572319 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/control_collection/terms/shard1 to Terms{values={core_node2=0}, version=0}
[junit4] 2> 2572322 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2572322 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2572322 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:38160/control_collection_shard1_replica_n1/
[junit4] 2> 2572323 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2572323 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:38160/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 2572327 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:38160/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 2572430 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2572433 INFO (qtp1830451017-17866) [n:127.0.0.1:38160_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1388
[junit4] 2> 2572436 INFO (qtp1830451017-17870) [n:127.0.0.1:38160_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 2572533 INFO (zkCallback-4847-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2572933 INFO (OverseerCollectionConfigSetProcessor-72347318965501956-127.0.0.1:38160_-n_0000000000) [n:127.0.0.1:38160_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2573437 INFO (qtp1830451017-17870) [n:127.0.0.1:38160_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:38160_&wt=javabin&version=2} status=0 QTime=2509
[junit4] 2> 2573443 INFO (zkConnectionManagerCallback-4866-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2573444 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2573445 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45253/solr ready
[junit4] 2> 2573445 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 2573446 INFO (qtp1830451017-17865) [n:127.0.0.1:38160_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2573450 INFO (OverseerThreadFactory-6750-thread-2-processing-n:127.0.0.1:38160_) [n:127.0.0.1:38160_ ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 2573655 WARN (OverseerThreadFactory-6750-thread-2-processing-n:127.0.0.1:38160_) [n:127.0.0.1:38160_ ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 2573658 INFO (qtp1830451017-17865) [n:127.0.0.1:38160_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 2573658 INFO (qtp1830451017-17865) [n:127.0.0.1:38160_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=212
[junit4] 2> 2573846 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_8D41AC50B277880A-001/shard-1-001 of type NRT
[junit4] 2> 2573846 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_191-b12
[junit4] 2> 2573919 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2573920 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2573920 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 2573931 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@33ed9f95{/,null,AVAILABLE}
[junit4] 2> 2573931 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3c6d86a4{HTTP/1.1,[http/1.1]}{127.0.0.1:40634}
[junit4] 2> 2573932 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.e.j.s.Server Started @2573994ms
[junit4] 2> 2573932 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_8D41AC50B277880A-001/tempDir-001/jetty1, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=40634, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_8D41AC50B277880A-001/shard-1-001/cores}
[junit4] 2> 2573932 ERROR (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2573932 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2573932 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 2573932 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2573932 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2573932 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-11-27T02:34:33.212Z
[junit4] 2> 2573980 INFO (zkConnectionManagerCallback-4868-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2573981 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2573981 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_8D41AC50B277880A-001/shard-1-001/solr.xml
[junit4] 2> 2573987 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2573987 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2573989 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@19988b7a, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2574294 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45253/solr
[junit4] 2> 2574296 INFO (zkConnectionManagerCallback-4872-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2574299 INFO (zkConnectionManagerCallback-4874-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2574310 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2574315 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2574315 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40634_
[junit4] 2> 2574320 INFO (zkCallback-4865-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2574321 INFO (zkCallback-4847-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2574324 INFO (zkCallback-4854-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2574335 INFO (zkCallback-4873-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2574344 INFO (zkConnectionManagerCallback-4881-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2574345 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2574347 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45253/solr ready
[junit4] 2> 2574347 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 2574377 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19988b7a
[junit4] 2> 2574394 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19988b7a
[junit4] 2> 2574394 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19988b7a
[junit4] 2> 2574396 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_8D41AC50B277880A-001/shard-1-001/cores
[junit4] 2> 2574437 INFO (qtp1830451017-17871) [n:127.0.0.1:38160_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:40634_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2574441 INFO (OverseerCollectionConfigSetProcessor-72347318965501956-127.0.0.1:38160_-n_0000000000) [n:127.0.0.1:38160_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2574445 INFO (qtp1830451017-17869) [n:127.0.0.1:38160_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.control_collection.shard1.replica_n1:INDEX.sizeInBytes} status=0 QTime=1
[junit4] 2> 2574446 INFO (qtp1830451017-17870) [n:127.0.0.1:38160_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2574448 INFO (qtp163034312-17925) [n:127.0.0.1:40634_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2574448 INFO (OverseerThreadFactory-6750-thread-3-processing-n:127.0.0.1:38160_) [n:127.0.0.1:38160_ c:collection1 s:shard1 ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:40634_ for creating new replica of shard shard1 for collection collection1
[junit4] 2> 2574453 INFO (qtp163034312-17920) [n:127.0.0.1:40634_ x:collection1_shard1_replica_n21] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n21&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2575473 INFO (qtp163034312-17920) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2575489 INFO (qtp163034312-17920) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.IndexSchema [collection1_shard1_replica_n21] Schema name=test
[junit4] 2> 2575595 INFO (qtp163034312-17920) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 2575613 INFO (qtp163034312-17920) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n21' using configuration from collection collection1, trusted=true
[junit4] 2> 2575614 INFO (qtp163034312-17920) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n21' (registry 'solr.core.collection1.shard1.replica_n21') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19988b7a
[junit4] 2> 2575614 INFO (qtp163034312-17920) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrCore [[collection1_shard1_replica_n21] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_8D41AC50B277880A-001/shard-1-001/cores/collection1_shard1_replica_n21], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_8D41AC50B277880A-001/shard-1-001/cores/collection1_shard1_replica_n21/data/]
[junit4] 2> 2575618 INFO (qtp163034312-17920) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@a94d06f
[junit4] 2> 2575626 WARN (qtp163034312-17920) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2575688 INFO (qtp163034312-17920) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2575688 INFO (qtp163034312-17920) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2575689 INFO (qtp163034312-17920) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.CommitTracker Hard AutoCommit: if uncommitted for 30000ms;
[junit4] 2> 2575689 INFO (qtp163034312-17920) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.CommitTracker Soft AutoCommit: if uncommitted for 3000ms;
[junit4] 2> 2575691 INFO (qtp163034312-17920) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=41, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 2575691 INFO (qtp163034312-17920) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.SolrIndexSearcher Opening [Searcher@6ebd7794[collection1_shard1_replica_n21] main]
[junit4] 2> 2575693 INFO (qtp163034312-17920) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2575693 INFO (qtp163034312-17920) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2575694 INFO (qtp163034312-17920) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2575695 INFO (searcherExecutor-6769-thread-1-processing-n:127.0.0.1:40634_ x:collection1_shard1_replica_n21 c:collection1 s:shard1) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrCore [collection1_shard1_replica_n21] Registered new searcher Searcher@6ebd7794[collection1_shard1_replica_n21] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2575695 INFO (qtp163034312-17920) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1618252739352985600
[junit4] 2> 2575702 INFO (qtp163034312-17920) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node22=0}, version=0}
[junit4] 2> 2575706 INFO (qtp163034312-17920) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2575706 INFO (qtp163034312-17920) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2575706 INFO (qtp163034312-17920) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40634/collection1_shard1_replica_n21/
[junit4] 2> 2575707 INFO (qtp163034312-17920) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2575707 INFO (qtp163034312-17920) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SyncStrategy http://127.0.0.1:40634/collection1_shard1_replica_n21/ has no replicas
[junit4] 2> 2575711 INFO (qtp163034312-17920) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40634/collection1_shard1_replica_n21/ shard1
[junit4] 2> 2575862 INFO (qtp163034312-17920) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2575865 INFO (qtp163034312-17920) [n:127.0.0.1:40634_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n21&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1413
[junit4] 2> 2575869 INFO (qtp1830451017-17871) [n:127.0.0.1:38160_ c:collection1 ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:40634_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=1432
[junit4] 2> 2576062 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_8D41AC50B277880A-001/shard-2-001 of type NRT
[junit4] 2> 2576063 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_191-b12
[junit4] 2> 2576064 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2576064 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2576065 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 2576065 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@547c5f29{/,null,AVAILABLE}
[junit4] 2> 2576066 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@38bbbe60{HTTP/1.1,[http/1.1]}{127.0.0.1:35735}
[junit4] 2> 2576071 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.e.j.s.Server Started @2576133ms
[junit4] 2> 2576071 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_8D41AC50B277880A-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/, hostPort=35735, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_8D41AC50B277880A-001/shard-2-001/cores}
[junit4] 2> 2576071 ERROR (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2576073 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2576073 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 2576073 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2576073 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2576074 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-11-27T02:34:35.354Z
[junit4] 2> 2576076 INFO (zkConnectionManagerCallback-4884-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2576079 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2576079 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_8D41AC50B277880A-001/shard-2-001/solr.xml
[junit4] 2> 2576085 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2576085 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2576087 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@19988b7a, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2576210 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45253/solr
[junit4] 2> 2576212 INFO (zkConnectionManagerCallback-4888-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2576214 INFO (zkConnectionManagerCallback-4890-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2576227 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2576235 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2576235 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35735_
[junit4] 2> 2576237 INFO (zkCallback-4854-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2576243 INFO (zkCallback-4880-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2576243 INFO (zkCallback-4873-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2576243 INFO (zkCallback-4847-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2576243 INFO (zkCallback-4865-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2576244 INFO (zkCallback-4889-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2576267 INFO (zkConnectionManagerCallback-4897-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2576270 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2576275 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45253/solr ready
[junit4] 2> 2576275 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 2576306 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19988b7a
[junit4] 2> 2576324 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19988b7a
[junit4] 2> 2576324 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19988b7a
[junit4] 2> 2576327 INFO (TEST-RestartWhileUpdatingTest.test-seed#[8D41AC50B277880A]) [ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_8D41AC50B277880A-001/shard-2-001/cores
[junit4] 2> 2576384 INFO (qtp457683858-17962) [n:127.0.0.1:35735_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:35735_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2576392 INFO (OverseerCollectionConfigSetProcessor-72347318965501956-127.0.0.1:38160_-n_0000000000) [n:127.0.0.1:38160_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2576400 INFO (qtp457683858-17957) [n:127.0.0.1:35735_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2576403 INFO (qtp1830451017-17869) [n:127.0.0.1:38160_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.control_collection.shard1.replica_n1:INDEX.sizeInBytes} status=0 QTime=0
[junit4] 2> 2576405 INFO (qtp1830451017-17869) [n:127.0.0.1:38160_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2576407 INFO (qtp163034312-17921) [n:127.0.0.1:40634_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.collection1.shard1.replica_n21:INDEX.sizeInBytes} status=0 QTime=1
[junit4] 2> 2576436 INFO (qtp163034312-17927) [n:127.0.0.1:40634_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=26
[junit4] 2> 2576439 INFO (qtp163034312-17923) [n:127.0.0.1:40634_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.collection1.shard1.replica_n21:INDEX.sizeInBytes} status=0 QTime=0
[junit4] 2> 2576442 INFO (OverseerThreadFactory-6750-thread-4-processing-n:127.0.0.1:38160_) [n:127.0.0.1:38160_ c:collection1 s:shard1 ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:35735_ for creating new replica of shard shard1 for collection collection1
[junit4] 2> 2576449 INFO (qtp457683858-17958) [n:127.0.0.1:35735_ x:collection1_shard1_replica_n23] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n23&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2577471 INFO (qtp457683858-17958) [n:127.0.0.1:35735_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2577488 INFO (qtp457683858-17958) [n:127.0.0.1:35735_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.s.IndexSchema [collection1_shard1_replica_n23] Schema name=test
[junit4] 2> 2577590 INFO (qtp457683858-17958) [n:127.0.0.1:35735_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 2577622 INFO (qtp457683858-17958) [n:127.0.0.1:35735_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n23' using configuration from collection collection1, trusted=true
[junit4] 2> 2577623 INFO (qtp457683858-17958) [n:127.0.0.1:35735_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n23' (registry 'solr.core.collection1.shard1.replica_n23') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19988b7a
[junit4] 2> 2577623 INFO (qtp457683858-17958) [n:127.0.0.1:35735_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.SolrCore [[collection1_shard1_replica_n23] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_8D41AC50B277880A-001/shard-2-001/cores/collection1_shard1_replica_n23], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_8D41AC50B277880A-001/shard-2-001/cores/collection1_shard1_replica_n23/data/]
[junit4] 2> 2577627 INFO (qtp457683858-17958) [n:127.0.0.1:35735_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@4e501e18
[junit4] 2> 2577636 WARN (qtp457683858-17958) [n:127.0.0.1:35735_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2577697 INFO (qtp457683858-17958) [n:127.0.0.1:35735_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2577698 INFO (qtp457683858-17958) [n:127.0.0.1:35735_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2577699 INFO (qtp457683858-17958) [n:127.0.0.1:35735_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.CommitTracker Hard AutoCommit: if uncommitted for 30000ms;
[junit4] 2> 2577699 INFO (qtp457683858-17958) [n:127.0.0.1:35735_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.CommitTracker Soft AutoCommit: if uncommitted for 3000ms;
[junit4] 2> 2577701 INFO (qtp457683858-17958) [n:127.0.0.1:35735_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=41, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 2577701 INFO (qtp457683858-17958) [n:127.0.0.1:35735_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.s.SolrIndexSearcher Opening [Searcher@509ca7f1[collection1_shard1_replica_n23] main]
[junit4] 2> 2577703 INFO (qtp457683858-17958) [n:127.0.0.1:35735_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2577703 INFO (qtp457683858-17958) [n:127.0.0.1:35735_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2577704 INFO (qtp457683858-17958) [n:127.0.0.1:35735_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2577704 INFO (qtp457683858-17958) [n:127.0.0.1:35735_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1618252741459574784
[junit4] 2> 2577707 INFO (searcherExecutor-6783-thread-1-processing-n:127.0.0.1:35735_ x:collection1_shard1_replica_n23 c:collection1 s:shard1) [n:127.0.0.1:35735_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.SolrCore [collection1_shard1_replica_n23] Registered new searcher Searcher@509ca7f1[collection1_shard1_replica_n23] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2577710 INFO (qtp457683858-17958) [n:127.0.0.1:35735_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node24=0, core_node22=0}, version=1}
[junit4] 2> 2577712 INFO (qtp457683858-17958) [n:127.0.0.1:35735_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.ZkController Core needs to recover:collection1_shard1_replica_n23
[junit4] 2> 2577713 INFO (updateExecutor-4885-thread-1-processing-n:127.0.0.1:35735_ x:collection1_shard1_replica_n23 c:collection1 s:shard1) [n:127.0.0.1:35735_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 2577714 INFO (qtp457683858-17958) [n:127.0.0.1:35735_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n23&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1265
[junit4] 2> 2577715 INFO (recoveryExecutor-4886-thread-1-processing-n:127.0.0.1:35735_ x:collection1_shard1_replica_n23 c:collection1 s:shard1 r:core_node24) [n:127.0.0.1:35735_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 2577717 INFO (recoveryExecutor-4886-thread-1-processing-n:127.0.0.1:35735_ x:collection1_shard1_replica_n23 c:collection1 s:shard1 r:core_node24) [n:127.0.0.1:35735_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy startu
[...truncated too long message...]
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/null1767248217
[copy] Copying 240 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/null1767248217
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/null1767248217
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 48ms :: artifacts dl 3ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 291 minutes 16 seconds
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
at hudson.FilePath$34.hasMatch(FilePath.java:2678)
at hudson.FilePath$34.invoke(FilePath.java:2557)
at hudson.FilePath$34.invoke(FilePath.java:2547)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:955)
at hudson.FilePath.act(FilePath.java:1036)
at hudson.FilePath.act(FilePath.java:1025)
at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1819)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused: hudson.FilePath$TunneledInterruptedException
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2920)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
at hudson.FilePath.act(FilePath.java:1038)
at hudson.FilePath.act(FilePath.java:1025)
at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1819)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1706 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1706/
16 tests failed.
FAILED: org.apache.lucene.codecs.lucene50.TestLucene50TermVectorsFormat.testHighFreqs
Error Message:
Java heap space
Stack Trace:
java.lang.OutOfMemoryError: Java heap space
FAILED: org.apache.lucene.index.TestBinaryDocValuesUpdates.testTonsOfUpdates
Error Message:
Java heap space
Stack Trace:
java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([7C2E6A17C0EC5A10:40BB41C22CC75F2]:0)
at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:84)
at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:57)
at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:168)
at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:154)
at org.apache.lucene.codecs.lucene70.Lucene70DocValuesConsumer.addBinaryField(Lucene70DocValuesConsumer.java:348)
at org.apache.lucene.codecs.asserting.AssertingDocValuesFormat$AssertingDocValuesConsumer.addBinaryField(AssertingDocValuesFormat.java:107)
at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.addBinaryField(PerFieldDocValuesFormat.java:114)
at org.apache.lucene.index.ReadersAndUpdates.handleDVUpdates(ReadersAndUpdates.java:330)
at org.apache.lucene.index.ReadersAndUpdates.writeFieldUpdates(ReadersAndUpdates.java:570)
at org.apache.lucene.index.IndexWriter.writeSomeDocValuesUpdates(IndexWriter.java:628)
at org.apache.lucene.index.FrozenBufferedUpdates.forceApply(FrozenBufferedUpdates.java:322)
at org.apache.lucene.index.FrozenBufferedUpdates.tryApply(FrozenBufferedUpdates.java:228)
at org.apache.lucene.index.IndexWriter.lambda$publishFrozenUpdates$3(IndexWriter.java:2613)
at org.apache.lucene.index.IndexWriter$$Lambda$92/851977769.process(Unknown Source)
at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5136)
at org.apache.lucene.index.IndexWriter.updateDocValues(IndexWriter.java:1794)
at org.apache.lucene.index.TestBinaryDocValuesUpdates.testTonsOfUpdates(TestBinaryDocValuesUpdates.java:1324)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
FAILED: org.apache.lucene.index.TestBinaryDocValuesUpdates.testSimple
Error Message:
Captured an uncaught exception in thread: Thread[id=1059, name=Lucene Merge Thread #1, state=RUNNABLE, group=TGRP-TestBinaryDocValuesUpdates]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1059, name=Lucene Merge Thread #1, state=RUNNABLE, group=TGRP-TestBinaryDocValuesUpdates]
at __randomizedtesting.SeedInfo.seed([7C2E6A17C0EC5A10:449D4EE9E71F8EC1]:0)
Caused by: org.apache.lucene.index.MergePolicy$MergeException: org.apache.lucene.store.AlreadyClosedException: refusing to delete any files: this IndexWriter hit an unrecoverable exception
at __randomizedtesting.SeedInfo.seed([7C2E6A17C0EC5A10]:0)
at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:704)
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)
Caused by: org.apache.lucene.store.AlreadyClosedException: refusing to delete any files: this IndexWriter hit an unrecoverable exception
at org.apache.lucene.index.IndexFileDeleter.ensureOpen(IndexFileDeleter.java:349)
at org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:669)
at org.apache.lucene.index.IndexFileDeleter.deleteNewFiles(IndexFileDeleter.java:664)
at org.apache.lucene.index.IndexWriter.deleteNewFiles(IndexWriter.java:5054)
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4569)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4078)
at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:625)
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:662)
Caused by: java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([7C2E6A17C0EC5A10:40BB41C22CC75F2]:0)
at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:84)
at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:57)
at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:168)
at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:154)
at org.apache.lucene.codecs.lucene70.Lucene70DocValuesConsumer.addBinaryField(Lucene70DocValuesConsumer.java:348)
at org.apache.lucene.codecs.asserting.AssertingDocValuesFormat$AssertingDocValuesConsumer.addBinaryField(AssertingDocValuesFormat.java:107)
at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.addBinaryField(PerFieldDocValuesFormat.java:114)
at org.apache.lucene.index.ReadersAndUpdates.handleDVUpdates(ReadersAndUpdates.java:330)
at org.apache.lucene.index.ReadersAndUpdates.writeFieldUpdates(ReadersAndUpdates.java:570)
at org.apache.lucene.index.IndexWriter.writeSomeDocValuesUpdates(IndexWriter.java:628)
at org.apache.lucene.index.FrozenBufferedUpdates.forceApply(FrozenBufferedUpdates.java:322)
at org.apache.lucene.index.FrozenBufferedUpdates.tryApply(FrozenBufferedUpdates.java:228)
at org.apache.lucene.index.IndexWriter.lambda$publishFrozenUpdates$3(IndexWriter.java:2613)
at org.apache.lucene.index.IndexWriter$$Lambda$92/851977769.process(Unknown Source)
at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5136)
at org.apache.lucene.index.IndexWriter.updateDocValues(IndexWriter.java:1794)
at org.apache.lucene.index.TestBinaryDocValuesUpdates.testTonsOfUpdates(TestBinaryDocValuesUpdates.java:1324)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
FAILED: org.apache.lucene.index.TestDemoParallelLeafReader.testRandom
Error Message:
Java heap space
Stack Trace:
java.lang.OutOfMemoryError: Java heap space
FAILED: org.apache.lucene.index.TestIndexWriterMaxDocs.testAddTooManyIndexesDir
Error Message:
Java heap space
Stack Trace:
java.lang.OutOfMemoryError: Java heap space
FAILED: org.apache.lucene.index.TestIndexWriterWithThreads.testCloseWithThreads
Error Message:
this writer hit an unrecoverable error; cannot commit
Stack Trace:
java.lang.IllegalStateException: this writer hit an unrecoverable error; cannot commit
at __randomizedtesting.SeedInfo.seed([7C2E6A17C0EC5A10:61EA4360A31940B6]:0)
at org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4759)
at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:3298)
at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3466)
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3431)
at org.apache.lucene.index.TestIndexWriterWithThreads.testCloseWithThreads(TestIndexWriterWithThreads.java:241)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Suppressed: org.apache.lucene.store.AlreadyClosedException: refusing to delete any files: this IndexWriter hit an unrecoverable exception
at org.apache.lucene.index.IndexFileDeleter.ensureOpen(IndexFileDeleter.java:349)
at org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:669)
at org.apache.lucene.index.IndexFileDeleter.decRef(IndexFileDeleter.java:589)
at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:3308)
... 39 more
Caused by: java.lang.OutOfMemoryError: Java heap space
[CIRCULAR REFERENCE:java.lang.OutOfMemoryError: Java heap space]
FAILED: org.apache.lucene.index.TestIndexingSequenceNumbers.testStressConcurrentAddAndDeleteAndCommit
Error Message:
this IndexWriter is closed
Stack Trace:
org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:681)
at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:695)
at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3430)
at org.apache.lucene.index.TestIndexingSequenceNumbers.testStressConcurrentAddAndDeleteAndCommit(TestIndexingSequenceNumbers.java:547)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([7C2E6A17C0EC5A10]:0)
at org.apache.lucene.index.DocumentsWriterPerThread$IntBlockAllocator.getIntBlock(DocumentsWriterPerThread.java:662)
at org.apache.lucene.util.IntBlockPool.nextBuffer(IntBlockPool.java:155)
at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:159)
at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:856)
at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:441)
at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:405)
at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:251)
at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:494)
at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1618)
at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1237)
at org.apache.lucene.index.TestIndexingSequenceNumbers$4.run(TestIndexingSequenceNumbers.java:526)
FAILED: org.apache.lucene.index.TestMixedDocValuesUpdates.testTonsOfUpdates
Error Message:
Java heap space
Stack Trace:
java.lang.OutOfMemoryError: Java heap space
FAILED: org.apache.lucene.index.TestNumericDocValuesUpdates.testTonsOfUpdates
Error Message:
Java heap space: failed reallocation of scalar replaced objects
Stack Trace:
java.lang.OutOfMemoryError: Java heap space: failed reallocation of scalar replaced objects
FAILED: org.apache.lucene.index.TestTermsEnum.testIntersectRandom
Error Message:
Java heap space
Stack Trace:
java.lang.OutOfMemoryError: Java heap space
FAILED: org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica
Error Message:
Expected new active leader null Live Nodes: [127.0.0.1:39144_solr, 127.0.0.1:40378_solr, 127.0.0.1:41779_solr] Last available state: DocCollection(raceDeleteReplica_false//collections/raceDeleteReplica_false/state.json/14)={ "pullReplicas":"0", "replicationFactor":"2", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{ "core_node4":{ "core":"raceDeleteReplica_false_shard1_replica_n2", "base_url":"http://127.0.0.1:46158/solr", "node_name":"127.0.0.1:46158_solr", "state":"down", "type":"NRT", "force_set_state":"false", "leader":"true"}, "core_node6":{ "core":"raceDeleteReplica_false_shard1_replica_n5", "base_url":"http://127.0.0.1:46158/solr", "node_name":"127.0.0.1:46158_solr", "state":"down", "type":"NRT", "force_set_state":"false"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0"}
Stack Trace:
java.lang.AssertionError: Expected new active leader
null
Live Nodes: [127.0.0.1:39144_solr, 127.0.0.1:40378_solr, 127.0.0.1:41779_solr]
Last available state: DocCollection(raceDeleteReplica_false//collections/raceDeleteReplica_false/state.json/14)={
"pullReplicas":"0",
"replicationFactor":"2",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{
"core_node4":{
"core":"raceDeleteReplica_false_shard1_replica_n2",
"base_url":"http://127.0.0.1:46158/solr",
"node_name":"127.0.0.1:46158_solr",
"state":"down",
"type":"NRT",
"force_set_state":"false",
"leader":"true"},
"core_node6":{
"core":"raceDeleteReplica_false_shard1_replica_n5",
"base_url":"http://127.0.0.1:46158/solr",
"node_name":"127.0.0.1:46158_solr",
"state":"down",
"type":"NRT",
"force_set_state":"false"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0"}
at __randomizedtesting.SeedInfo.seed([1C427C2D2F9B9AF5:76541DFD4769D03F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
at org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica(DeleteReplicaTest.java:328)
at org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica(DeleteReplicaTest.java:224)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.DeleteReplicaTest.deleteReplicaByCountForAllShards
Error Message:
Error from server at http://127.0.0.1:39144/solr: Cannot create collection deleteByCountNew. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 3. This allows a maximum of 3 to be created. Value of numShards is 2, value of nrtReplicas is 2, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 4 shards to be created (higher than the allowed number)
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:39144/solr: Cannot create collection deleteByCountNew. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 3. This allows a maximum of 3 to be created. Value of numShards is 2, value of nrtReplicas is 2, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 4 shards to be created (higher than the allowed number)
at __randomizedtesting.SeedInfo.seed([1C427C2D2F9B9AF5:20D33B7960DAC7A0]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.DeleteReplicaTest.deleteReplicaByCountForAllShards(DeleteReplicaTest.java:161)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest
Error Message:
Error from server at http://127.0.0.1:39144/solr: Cannot create collection delLiveColl. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 3. This allows a maximum of 3 to be created. Value of numShards is 2, value of nrtReplicas is 2, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 4 shards to be created (higher than the allowed number)
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:39144/solr: Cannot create collection delLiveColl. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 3. This allows a maximum of 3 to be created. Value of numShards is 2, value of nrtReplicas is 2, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 4 shards to be created (higher than the allowed number)
at __randomizedtesting.SeedInfo.seed([1C427C2D2F9B9AF5:B122C82632A43280]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest(DeleteReplicaTest.java:73)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestSimGenericDistributedQueue.testDistributedQueueBlocking
Error Message:
Stack Trace:
java.util.concurrent.TimeoutException
at __randomizedtesting.SeedInfo.seed([1C427C2D2F9B9AF5:59E80E546BC92681]:0)
at java.util.concurrent.FutureTask.get(FutureTask.java:205)
at org.apache.solr.cloud.autoscaling.sim.TestSimDistributedQueue.testDistributedQueueBlocking(TestSimDistributedQueue.java:102)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.autoscaling.sim.TestSimGenericDistributedQueue
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.autoscaling.sim.TestSimGenericDistributedQueue: 1) Thread[id=8220, name=sdqtest--2285-thread-1, state=TIMED_WAITING, group=TGRP-TestSimGenericDistributedQueue] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.apache.solr.cloud.autoscaling.sim.GenericDistributedQueue.peek(GenericDistributedQueue.java:194) at org.apache.solr.cloud.autoscaling.sim.GenericDistributedQueue.peek(GenericDistributedQueue.java:167) at org.apache.solr.cloud.autoscaling.sim.TestSimDistributedQueue.lambda$testDistributedQueueBlocking$0(TestSimDistributedQueue.java:93) at org.apache.solr.cloud.autoscaling.sim.TestSimDistributedQueue$$Lambda$947/1389395902.call(Unknown Source) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$10/963048326.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.autoscaling.sim.TestSimGenericDistributedQueue:
1) Thread[id=8220, name=sdqtest--2285-thread-1, state=TIMED_WAITING, group=TGRP-TestSimGenericDistributedQueue]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.apache.solr.cloud.autoscaling.sim.GenericDistributedQueue.peek(GenericDistributedQueue.java:194)
at org.apache.solr.cloud.autoscaling.sim.GenericDistributedQueue.peek(GenericDistributedQueue.java:167)
at org.apache.solr.cloud.autoscaling.sim.TestSimDistributedQueue.lambda$testDistributedQueueBlocking$0(TestSimDistributedQueue.java:93)
at org.apache.solr.cloud.autoscaling.sim.TestSimDistributedQueue$$Lambda$947/1389395902.call(Unknown Source)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$10/963048326.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([1C427C2D2F9B9AF5]:0)
FAILED: org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessorTest.testPreemptiveCreation
Error Message:
Error from server at http://127.0.0.1:43000/solr/myalias_2017-10-23_shard1_replica_n1: KeeperErrorCode = Session expired for /configs/timeConfigtestPreemptiveCreation
Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at http://127.0.0.1:43000/solr/myalias_2017-10-23_shard1_replica_n1: KeeperErrorCode = Session expired for /configs/timeConfigtestPreemptiveCreation
at __randomizedtesting.SeedInfo.seed([1C427C2D2F9B9AF5:7128F98617F977C1]:0)
at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:551)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1016)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:177)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:138)
at org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessorTest.testPreemptiveCreation(TimeRoutedAliasUpdateProcessorTest.java:351)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:43000/solr/myalias_2017-10-23_shard1_replica_n1: KeeperErrorCode = Session expired for /configs/timeConfigtestPreemptiveCreation
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:528)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Build Log:
[...truncated 811 lines...]
[junit4] Suite: org.apache.lucene.index.TestBinaryDocValuesUpdates
[junit4] 1> TEST: isNRT=false reader1=StandardDirectoryReader(segments_1:4 _0(8.0.0):C2)
[junit4] 1> TEST: now reopen
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestBinaryDocValuesUpdates -Dtests.method=testTonsOfUpdates -Dtests.seed=7C2E6A17C0EC5A10 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=en-US -Dtests.timezone=US/Michigan -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 28.9s J1 | TestBinaryDocValuesUpdates.testTonsOfUpdates <<<
[junit4] > Throwable #1: java.lang.OutOfMemoryError: Java heap space
[junit4] > at __randomizedtesting.SeedInfo.seed([7C2E6A17C0EC5A10:40BB41C22CC75F2]:0)
[junit4] > at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:84)
[junit4] > at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:57)
[junit4] > at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:168)
[junit4] > at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:154)
[junit4] > at org.apache.lucene.codecs.lucene70.Lucene70DocValuesConsumer.addBinaryField(Lucene70DocValuesConsumer.java:348)
[junit4] > at org.apache.lucene.codecs.asserting.AssertingDocValuesFormat$AssertingDocValuesConsumer.addBinaryField(AssertingDocValuesFormat.java:107)
[junit4] > at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.addBinaryField(PerFieldDocValuesFormat.java:114)
[junit4] > at org.apache.lucene.index.ReadersAndUpdates.handleDVUpdates(ReadersAndUpdates.java:330)
[junit4] > at org.apache.lucene.index.ReadersAndUpdates.writeFieldUpdates(ReadersAndUpdates.java:570)
[junit4] > at org.apache.lucene.index.IndexWriter.writeSomeDocValuesUpdates(IndexWriter.java:628)
[junit4] > at org.apache.lucene.index.FrozenBufferedUpdates.forceApply(FrozenBufferedUpdates.java:322)
[junit4] > at org.apache.lucene.index.FrozenBufferedUpdates.tryApply(FrozenBufferedUpdates.java:228)
[junit4] > at org.apache.lucene.index.IndexWriter.lambda$publishFrozenUpdates$3(IndexWriter.java:2613)
[junit4] > at org.apache.lucene.index.IndexWriter$$Lambda$92/851977769.process(Unknown Source)
[junit4] > at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5136)
[junit4] > at org.apache.lucene.index.IndexWriter.updateDocValues(IndexWriter.java:1794)
[junit4] > at org.apache.lucene.index.TestBinaryDocValuesUpdates.testTonsOfUpdates(TestBinaryDocValuesUpdates.java:1324)
[junit4] 2> Nov 25, 2018 4:20:47 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
[junit4] 2> WARNING: Uncaught exception in thread: Thread[Lucene Merge Thread #1,5,TGRP-TestBinaryDocValuesUpdates]
[junit4] 2> org.apache.lucene.index.MergePolicy$MergeException: org.apache.lucene.store.AlreadyClosedException: refusing to delete any files: this IndexWriter hit an unrecoverable exception
[junit4] 2> at __randomizedtesting.SeedInfo.seed([7C2E6A17C0EC5A10]:0)
[junit4] 2> at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:704)
[junit4] 2> at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)
[junit4] 2> Caused by: org.apache.lucene.store.AlreadyClosedException: refusing to delete any files: this IndexWriter hit an unrecoverable exception
[junit4] 2> at org.apache.lucene.index.IndexFileDeleter.ensureOpen(IndexFileDeleter.java:349)
[junit4] 2> at org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:669)
[junit4] 2> at org.apache.lucene.index.IndexFileDeleter.deleteNewFiles(IndexFileDeleter.java:664)
[junit4] 2> at org.apache.lucene.index.IndexWriter.deleteNewFiles(IndexWriter.java:5054)
[junit4] 2> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4569)
[junit4] 2> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4078)
[junit4] 2> at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:625)
[junit4] 2> at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:662)
[junit4] 2> Caused by: java.lang.OutOfMemoryError: Java heap space
[junit4] 2> at __randomizedtesting.SeedInfo.seed([7C2E6A17C0EC5A10:40BB41C22CC75F2]:0)
[junit4] 2> at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:84)
[junit4] 2> at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:57)
[junit4] 2> at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:168)
[junit4] 2> at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:154)
[junit4] 2> at org.apache.lucene.codecs.lucene70.Lucene70DocValuesConsumer.addBinaryField(Lucene70DocValuesConsumer.java:348)
[junit4] 2> at org.apache.lucene.codecs.asserting.AssertingDocValuesFormat$AssertingDocValuesConsumer.addBinaryField(AssertingDocValuesFormat.java:107)
[junit4] 2> at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.addBinaryField(PerFieldDocValuesFormat.java:114)
[junit4] 2> at org.apache.lucene.index.ReadersAndUpdates.handleDVUpdates(ReadersAndUpdates.java:330)
[junit4] 2> at org.apache.lucene.index.ReadersAndUpdates.writeFieldUpdates(ReadersAndUpdates.java:570)
[junit4] 2> at org.apache.lucene.index.IndexWriter.writeSomeDocValuesUpdates(IndexWriter.java:628)
[junit4] 2> at org.apache.lucene.index.FrozenBufferedUpdates.forceApply(FrozenBufferedUpdates.java:322)
[junit4] 2> at org.apache.lucene.index.FrozenBufferedUpdates.tryApply(FrozenBufferedUpdates.java:228)
[junit4] 2> at org.apache.lucene.index.IndexWriter.lambda$publishFrozenUpdates$3(IndexWriter.java:2613)
[junit4] 2> at org.apache.lucene.index.IndexWriter$$Lambda$92/851977769.process(Unknown Source)
[junit4] 2> at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5136)
[junit4] 2> at org.apache.lucene.index.IndexWriter.updateDocValues(IndexWriter.java:1794)
[junit4] 2> at org.apache.lucene.index.TestBinaryDocValuesUpdates.testTonsOfUpdates(TestBinaryDocValuesUpdates.java:1324)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2>
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestBinaryDocValuesUpdates -Dtests.method=testSimple -Dtests.seed=7C2E6A17C0EC5A10 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=en-US -Dtests.timezone=US/Michigan -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.05s J1 | TestBinaryDocValuesUpdates.testSimple <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1059, name=Lucene Merge Thread #1, state=RUNNABLE, group=TGRP-TestBinaryDocValuesUpdates]
[junit4] > at __randomizedtesting.SeedInfo.seed([7C2E6A17C0EC5A10:449D4EE9E71F8EC1]:0)
[junit4] > Caused by: org.apache.lucene.index.MergePolicy$MergeException: org.apache.lucene.store.AlreadyClosedException: refusing to delete any files: this IndexWriter hit an unrecoverable exception
[junit4] > at __randomizedtesting.SeedInfo.seed([7C2E6A17C0EC5A10]:0)
[junit4] > at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:704)
[junit4] > at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)
[junit4] > Caused by: org.apache.lucene.store.AlreadyClosedException: refusing to delete any files: this IndexWriter hit an unrecoverable exception
[junit4] > at org.apache.lucene.index.IndexFileDeleter.ensureOpen(IndexFileDeleter.java:349)
[junit4] > at org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:669)
[junit4] > at org.apache.lucene.index.IndexFileDeleter.deleteNewFiles(IndexFileDeleter.java:664)
[junit4] > at org.apache.lucene.index.IndexWriter.deleteNewFiles(IndexWriter.java:5054)
[junit4] > at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4569)
[junit4] > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4078)
[junit4] > at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:625)
[junit4] > at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:662)
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap space
[junit4] > at __randomizedtesting.SeedInfo.seed([7C2E6A17C0EC5A10:40BB41C22CC75F2]:0)
[junit4] > at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:84)
[junit4] > at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:57)
[junit4] > at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:168)
[junit4] > at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:154)
[junit4] > at org.apache.lucene.codecs.lucene70.Lucene70DocValuesConsumer.addBinaryField(Lucene70DocValuesConsumer.java:348)
[junit4] > at org.apache.lucene.codecs.asserting.AssertingDocValuesFormat$AssertingDocValuesConsumer.addBinaryField(AssertingDocValuesFormat.java:107)
[junit4] > at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.addBinaryField(PerFieldDocValuesFormat.java:114)
[junit4] > at org.apache.lucene.index.ReadersAndUpdates.handleDVUpdates(ReadersAndUpdates.java:330)
[junit4] > at org.apache.lucene.index.ReadersAndUpdates.writeFieldUpdates(ReadersAndUpdates.java:570)
[junit4] > at org.apache.lucene.index.IndexWriter.writeSomeDocValuesUpdates(IndexWriter.java:628)
[junit4] > at org.apache.lucene.index.FrozenBufferedUpdates.forceApply(FrozenBufferedUpdates.java:322)
[junit4] > at org.apache.lucene.index.FrozenBufferedUpdates.tryApply(FrozenBufferedUpdates.java:228)
[junit4] > at org.apache.lucene.index.IndexWriter.lambda$publishFrozenUpdates$3(IndexWriter.java:2613)
[junit4] > at org.apache.lucene.index.IndexWriter$$Lambda$92/851977769.process(Unknown Source)
[junit4] > at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5136)
[junit4] > at org.apache.lucene.index.IndexWriter.updateDocValues(IndexWriter.java:1794)
[junit4] > at org.apache.lucene.index.TestBinaryDocValuesUpdates.testTonsOfUpdates(TestBinaryDocValuesUpdates.java:1324)
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/build/core/test/J1/temp/lucene.index.TestBinaryDocValuesUpdates_7C2E6A17C0EC5A10-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene80): {bdv=PostingsFormat(name=LuceneVarGapDocFreqInterval), f=PostingsFormat(name=Direct), k1=PostingsFormat(name=Direct), foo=PostingsFormat(name=Direct), dvUpdateKey=PostingsFormat(name=LuceneVarGapDocFreqInterval), k2=PostingsFormat(name=LuceneFixedGap), upd=FST50, updKey=PostingsFormat(name=Direct), id=PostingsFormat(name=LuceneFixedGap), key=PostingsFormat(name=LuceneFixedGap)}, docValues:{ndv=DocValuesFormat(name=Lucene70), f10=DocValuesFormat(name=Direct), f12=DocValuesFormat(name=Lucene70), f11=DocValuesFormat(name=Asserting), f14=DocValuesFormat(name=Direct), f13=DocValuesFormat(name=Lucene70), f0=DocValuesFormat(name=Asserting), f16=DocValuesFormat(name=Lucene70), f1=DocValuesFormat(name=Lucene70), f15=DocValuesFormat(name=Asserting), f2=DocValuesFormat(name=Lucene70), f18=DocValuesFormat(name=Direct), f17=DocValuesFormat(name=Lucene70), f3=DocValuesFormat(name=Direct), f4=DocValuesFormat(name=Asserting), f19=DocValuesFormat(name=Asserting), f5=DocValuesFormat(name=Lucene70), bdv2=DocValuesFormat(name=Asserting), f6=DocValuesFormat(name=Lucene70), number=DocValuesFormat(name=Lucene70), f7=DocValuesFormat(name=Direct), f8=DocValuesFormat(name=Asserting), bdv1=DocValuesFormat(name=Direct), f9=DocValuesFormat(name=Lucene70), id=DocValuesFormat(name=Lucene70), val=DocValuesFormat(name=Direct), f21=DocValuesFormat(name=Direct), f20=DocValuesFormat(name=Lucene70), f23=DocValuesFormat(name=Lucene70), f22=DocValuesFormat(name=Asserting), f25=DocValuesFormat(name=Direct), upd=DocValuesFormat(name=Direct), f24=DocValuesFormat(name=Lucene70), f27=DocValuesFormat(name=Lucene70), sort=DocValuesFormat(name=Asserting), f26=DocValuesFormat(name=Asserting), cf0=DocValuesFormat(name=Direct), f28=DocValuesFormat(name=Lucene70), cf2=DocValuesFormat(name=Lucene70), cf1=DocValuesFormat(name=Asserting), cf4=DocValuesFormat(name=Direct), cf3=DocValuesFormat(name=Lucene70), cf6=DocValuesFormat(name=Lucene70), cf5=DocValuesFormat(name=Asserting), cf8=DocValuesFormat(name=Direct), cf7=DocValuesFormat(name=Lucene70), cf9=DocValuesFormat(name=Asserting), ssdv=DocValuesFormat(name=Asserting), sdv=DocValuesFormat(name=Direct), cf27=DocValuesFormat(name=Lucene70), cf28=DocValuesFormat(name=Direct), cf25=DocValuesFormat(name=Asserting), cf26=DocValuesFormat(name=Lucene70), cf23=DocValuesFormat(name=Lucene70), cf24=DocValuesFormat(name=Direct), cf21=DocValuesFormat(name=Asserting), cf22=DocValuesFormat(name=Lucene70), cf20=DocValuesFormat(name=Direct), key=DocValuesFormat(name=Lucene70), cf=DocValuesFormat(name=Lucene70), cf18=DocValuesFormat(name=Asserting), cf19=DocValuesFormat(name=Lucene70), f=DocValuesFormat(name=Asserting), cf16=DocValuesFormat(name=Lucene70), cf17=DocValuesFormat(name=Direct), cf14=DocValuesFormat(name=Asserting), cf15=DocValuesFormat(name=Lucene70), control=DocValuesFormat(name=Direct), cf12=DocValuesFormat(name=Lucene70), cf13=DocValuesFormat(name=Direct), cf10=DocValuesFormat(name=Asserting), cf11=DocValuesFormat(name=Lucene70), bdv=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=1648, maxMBSortInHeap=6.650856656555558, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@100c7579), locale=en-US, timezone=US/Michigan
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_191 (64-bit)/cpus=4,threads=1,free=69147952,total=444071936
[junit4] 2> NOTE: All tests run in this JVM: [TestSloppyPhraseQuery, TestDisjunctionMaxQuery, TestDocValuesRewriteMethod, TestTerm, TestRecyclingIntBlockAllocator, TestLongBitSet, TestMatchesIterator, Test2BBKDPoints, TestLongPostings, TestNotDocIdSet, TestSimpleFSLockFactory, TestThreadedForceMerge, TestSimilarityBase, TestTopDocsMerge, TestAttributeSource, TestVersion, Test2BSortedDocValuesFixedSorted, TestReadOnlyIndex, TestStressDeletes, TestSloppyMath, TestTrackingDirectoryWrapper, TestLockFactory, TestBytesRefArray, TestIndexWriterMergePolicy, TestAllFilesDetectTruncation, TestTimSorter, TestFutureArrays, TestGraphTokenizers, TestMaxTermFrequency, TestFilterSpans, TestIndexWriterFromReader, TestPagedBytes, TestFilterDirectory, TestAtomicUpdate, TestNIOFSDirectory, TestOneMergeWrappingMergePolicy, TestMutablePointsReaderUtils, TestSpansEnum, TestSynonymQuery, TestDirectMonotonic, TestSpanExplanationsOfNonMatches, TestTwoPhaseCommitTool, TestRegexpRandom, TestIndexTooManyDocs, TestLMDirichletSimilarity, TestWildcard, TestIndexWriterOnDiskFull, TestParallelLeafReader, TestDocsWithFieldSet, TestFieldInvertState, TestToken, TestInPlaceMergeSorter, TestNewestSegment, TestDemo, TestIndependenceChiSquared, TestLongValuesSource, TestPrefixRandom, TestIndexedDISI, TestCrashCausesCorruptIndex, TestBasicModelIn, TestOperations, TestBinaryDocValuesUpdates]
[junit4] Completed [146/511 (1!)] on J1 in 109.70s, 29 tests, 2 errors <<< FAILURES!
[...truncated 561 lines...]
[junit4] Suite: org.apache.lucene.index.TestIndexingSequenceNumbers
[junit4] 2> Nov 25, 2018 11:24:04 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
[junit4] 2> WARNING: Uncaught exception in thread: Thread[thread4,5,TGRP-TestIndexingSequenceNumbers]
[junit4] 2> java.lang.OutOfMemoryError: Java heap space
[junit4] 2> at __randomizedtesting.SeedInfo.seed([7C2E6A17C0EC5A10]:0)
[junit4] 2> at org.apache.lucene.index.DocumentsWriterPerThread$IntBlockAllocator.getIntBlock(DocumentsWriterPerThread.java:662)
[junit4] 2> at org.apache.lucene.util.IntBlockPool.nextBuffer(IntBlockPool.java:155)
[junit4] 2> at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:159)
[junit4] 2> at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:856)
[junit4] 2> at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:441)
[junit4] 2> at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:405)
[junit4] 2> at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:251)
[junit4] 2> at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:494)
[junit4] 2> at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1618)
[junit4] 2> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1237)
[junit4] 2> at org.apache.lucene.index.TestIndexingSequenceNumbers$4.run(TestIndexingSequenceNumbers.java:526)
[junit4] 2>
[junit4] 2> Nov 25, 2018 11:24:55 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
[junit4] 2> WARNING: Uncaught exception in thread: Thread[thread3,5,TGRP-TestIndexingSequenceNumbers]
[junit4] 2> java.lang.RuntimeException: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
[junit4] 2> at org.apache.lucene.index.TestIndexingSequenceNumbers$4.run(TestIndexingSequenceNumbers.java:534)
[junit4] 2> Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
[junit4] 2> at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:681)
[junit4] 2> at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:695)
[junit4] 2> at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1615)
[junit4] 2> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1237)
[junit4] 2> at org.apache.lucene.index.TestIndexingSequenceNumbers$4.run(TestIndexingSequenceNumbers.java:526)
[junit4] 2> Caused by: java.lang.OutOfMemoryError: Java heap space
[junit4] 2> at __randomizedtesting.SeedInfo.seed([7C2E6A17C0EC5A10]:0)
[junit4] 2> at org.apache.lucene.index.DocumentsWriterPerThread$IntBlockAllocator.getIntBlock(DocumentsWriterPerThread.java:662)
[junit4] 2> at org.apache.lucene.util.IntBlockPool.nextBuffer(IntBlockPool.java:155)
[junit4] 2> at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:159)
[junit4] 2> at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:856)
[junit4] 2> at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:441)
[junit4] 2> at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:405)
[junit4] 2> at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:251)
[junit4] 2> at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:494)
[junit4] 2> at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1618)
[junit4] 2> ... 2 more
[junit4] 2>
[junit4] 2> Nov 25, 2018 11:25:17 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
[junit4] 2> WARNING: Uncaught exception in thread: Thread[thread2,5,TGRP-TestIndexingSequenceNumbers]
[junit4] 2> java.lang.RuntimeException: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
[junit4] 2> at org.apache.lucene.index.TestIndexingSequenceNumbers$4.run(TestIndexingSequenceNumbers.java:534)
[junit4] 2> Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
[junit4] 2> at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:681)
[junit4] 2> at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:695)
[junit4] 2> at org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1305)
[junit4] 2> at org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1281)
[junit4] 2> at org.apache.lucene.index.TestIndexingSequenceNumbers$4.run(TestIndexingSequenceNumbers.java:524)
[junit4] 2> Caused by: java.lang.OutOfMemoryError: Java heap space
[junit4] 2> at __randomizedtesting.SeedInfo.seed([7C2E6A17C0EC5A10]:0)
[junit4] 2> at org.apache.lucene.index.DocumentsWriterPerThread$IntBlockAllocator.getIntBlock(DocumentsWriterPerThread.java:662)
[junit4] 2> at org.apache.lucene.util.IntBlockPool.nextBuffer(IntBlockPool.java:155)
[junit4] 2> at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:159)
[junit4] 2> at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:856)
[junit4] 2> at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:441)
[junit4] 2> at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:405)
[junit4] 2> at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:251)
[junit4] 2> at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:494)
[junit4] 2> at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1618)
[junit4] 2> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1237)
[junit4] 2> at org.apache.lucene.index.TestIndexingSequenceNumbers$4.run(TestIndexingSequenceNumbers.java:526)
[junit4] 2>
[junit4] 2> Nov 25, 2018 11:25:17 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
[junit4] 2> WARNING: Uncaught exception in thread: Thread[Lucene Merge Thread #1488,5,TGRP-TestIndexingSequenceNumbers]
[junit4] 2> java.lang.OutOfMemoryError: Java heap space
[junit4] 2>
[junit4] 2> Nov 25, 2018 11:25:13 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
[junit4] 2> WARNING: Uncaught exception in thread: Thread[Lucene Merge Thread #1489,5,TGRP-TestIndexingSequenceNumbers]
[junit4] 2> java.lang.OutOfMemoryError: Java heap space: failed reallocation of scalar replaced objects
[junit4] 2>
[junit4] 2> Nov 25, 2018 11:25:13 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
[junit4] 2> WARNING: Uncaught exception in thread: Thread[thread0,5,TGRP-TestIndexingSequenceNumbers]
[junit4] 2> java.lang.RuntimeException: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
[junit4] 2> at org.apache.lucene.index.TestIndexingSequenceNumbers$4.run(TestIndexingSequenceNumbers.java:534)
[junit4] 2> Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
[junit4] 2> at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:681)
[junit4] 2> at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:695)
[junit4] 2> at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3430)
[junit4] 2> at org.apache.lucene.index.TestIndexingSequenceNumbers$4.run(TestIndexingSequenceNumbers.java:502)
[junit4] 2> Caused by: java.lang.OutOfMemoryError: Java heap space
[junit4] 2> at __randomizedtesting.SeedInfo.seed([7C2E6A17C0EC5A10]:0)
[junit4] 2> at org.apache.lucene.index.DocumentsWriterPerThread$IntBlockAllocator.getIntBlock(DocumentsWriterPerThread.java:662)
[junit4] 2> at org.apache.lucene.util.IntBlockPool.nextBuffer(IntBlockPool.java:155)
[junit4] 2> at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:159)
[junit4] 2> at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:856)
[junit4] 2> at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:441)
[junit4] 2> at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:405)
[junit4] 2> at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:251)
[junit4] 2> at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:494)
[junit4] 2> at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1618)
[junit4] 2> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1237)
[junit4] 2> at org.apache.lucene.index.TestIndexingSequenceNumbers$4.run(TestIndexingSequenceNumbers.java:526)
[junit4] 2>
[junit4] 2> Nov 25, 2018 11:25:10 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
[junit4] 2> WARNING: Uncaught exception in thread: Thread[Lucene Merge Thread #1487,5,TGRP-TestIndexingSequenceNumbers]
[junit4] 2> org.apache.lucene.index.MergePolicy$MergeException: java.lang.IllegalStateException: this writer hit an unrecoverable error; cannot merge
[junit4] 2> at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:704)
[junit4] 2> at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)
[junit4] 2> Caused by: java.lang.IllegalStateException: this writer hit an unrecoverable error; cannot merge
[junit4] 2> at org.apache.lucene.index.IndexWriter._mergeInit(IndexWriter.java:4240)
[junit4] 2> at org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:4220)
[junit4] 2> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4072)
[junit4] 2> at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:625)
[junit4] 2> at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:662)
[junit4] 2> Caused by: java.lang.OutOfMemoryError: Java heap space
[junit4] 2> at __randomizedtesting.SeedInfo.seed([7C2E6A17C0EC5A10]:0)
[junit4] 2> at org.apache.lucene.index.DocumentsWriterPerThread$IntBlockAllocator.getIntBlock(DocumentsWriterPerThread.java:662)
[junit4] 2> at org.apache.lucene.util.IntBlockPool.nextBuffer(IntBlockPool.java:155)
[junit4] 2> at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:159)
[junit4] 2> at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:856)
[junit4] 2> at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:441)
[junit4] 2> at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:405)
[junit4] 2> at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:251)
[junit4] 2> at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:494)
[junit4] 2> at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1618)
[junit4] 2> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1237)
[junit4] 2> at org.apache.lucene.index.TestIndexingSequenceNumbers$4.run(TestIndexingSequenceNumbers.java:526)
[junit4] 2>
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestIndexingSequenceNumbers -Dtests.method=testStressConcurrentAddAndDeleteAndCommit -Dtests.seed=7C2E6A17C0EC5A10 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=en-AU -Dtests.timezone=Africa/Bangui -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 394s J1 | TestIndexingSequenceNumbers.testStressConcurrentAddAndDeleteAndCommit <<<
[junit4] > Throwable #1: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
[junit4] > at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:681)
[junit4] > at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:695)
[junit4] > at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3430)
[junit4] > at org.apache.lucene.index.TestIndexingSequenceNumbers.testStressConcurrentAddAndDeleteAndCommit(TestIndexingSequenceNumbers.java:547)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap space
[junit4] > at __randomizedtesting.SeedInfo.seed([7C2E6A17C0EC5A10]:0)
[junit4] > at org.apache.lucene.index.DocumentsWriterPerThread$IntBlockAllocator.getIntBlock(DocumentsWriterPerThread.java:662)
[junit4] > at org.apache.lucene.util.IntBlockPool.nextBuffer(IntBlockPool.java:155)
[junit4] > at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:159)
[junit4] > at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:856)
[junit4] > at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:441)
[junit4] > at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:405)
[junit4] > at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:251)
[junit4] > at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:494)
[junit4] > at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1618)
[junit4] > at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1237)
[junit4] > at org.apache.lucene.index.TestIndexingSequenceNumbers$4.run(TestIndexingSequenceNumbers.java:526)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3491, name=thread4, state=RUNNABLE, group=TGRP-TestIndexingSequenceNumbers]
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap space
[junit4] > at __randomizedtesting.SeedInfo.seed([7C2E6A17C0EC5A10]:0)
[junit4] > at org.apache.lucene.index.DocumentsWriterPerThread$IntBlockAllocator.getIntBlock(DocumentsWriterPerThread.java:662)
[junit4] > at org.apache.lucene.util.IntBlockPool.nextBuffer(IntBlockPool.java:155)
[junit4] > at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:159)
[junit4] > at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:856)
[junit4] > at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:441)
[junit4] > at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:405)
[junit4] > at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:251)
[junit4] > at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:494)
[junit4] > at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1618)
[junit4] > at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1237)
[junit4] > at org.apache.lucene.index.TestIndexingSequenceNumbers$4.run(TestIndexingSequenceNumbers.java:526)Throwable #3: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3488, name=thread1, state=RUNNABLE, group=TGRP-TestIndexingSequenceNumbers]
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap space
[junit4] > at __randomizedtesting.SeedInfo.seed([7C2E6A17C0EC5A10]:0)
[junit4] > at java.util.HashMap.resize(HashMap.java:704)
[junit4] > at java.util.HashMap.putVal(HashMap.java:663)
[junit4] > at java.util.HashMap.put(HashMap.java:612)
[junit4] > at java.util.HashSet.add(HashSet.java:220)
[junit4] > at java.util.AbstractCollection.addAll(AbstractCollection.java:344)
[junit4] > at org.apache.lucene.index.SegmentInfos.files(SegmentInfos.java:781)
[junit4] > at org.apache.lucene.index.IndexFileDeleter.incRef(IndexFileDeleter.java:550)
[junit4] > at org.apache.lucene.index.IndexFileDeleter.checkpoint(IndexFileDeleter.java:517)
[junit4] > at org.apache.lucene.index.IndexWriter.checkpointNoSIS(IndexWriter.java:2596)
[junit4] > at org.apache.lucene.index.IndexWriter.writeReaderPool(IndexWriter.java:3329)
[junit4] > at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:3239)
[junit4] > at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3466)
[junit4] > at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3431)
[junit4] > at org.apache.lucene.index.TestIndexingSequenceNumbers$4.run(TestIndexingSequenceNumbers.java:502)Throwable #4: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3490, name=thread3, state=RUNNABLE, group=TGRP-TestIndexingSequenceNumbers]
[junit4] > Caused by: java.lang.RuntimeException: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
[junit4] > at org.apache.lucene.index.TestIndexingSequenceNumbers$4.run(TestIndexingSequenceNumbers.java:534)
[junit4] > Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
[junit4] > at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:681)
[junit4] > at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:695)
[junit4] > at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1615)
[junit4] > at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1237)
[junit4] > at org.apache.lucene.index.TestIndexingSequenceNumbers$4.run(TestIndexingSequenceNumbers.java:526)
[junit4] > Caused by: java.lang.OutOfMemoryError: Java heap space
[junit4] > at __randomizedtesting.SeedInfo.seed([7C2E6A17C0EC5A10]:0)
[junit4] > at org.apache.lucene.index.DocumentsWriterPerThread$IntBlockAllocator.getIntBlock(DocumentsWriterPerThread.java:662)
[junit4] > at org.apache.lucene.util.IntBlockPool.nextBuffer(IntBlockPool.java:155)
[junit4] > at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:159)
[junit4] > at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:856)
[junit4] > at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:441)
[junit4] > at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:405)
[junit4] > at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:251)
[junit4] > at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:494)
[junit4] > at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1618)
[junit4] > ... 2 moreThrowable #5: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=4979, name=Lucene Merge Thread #1487, state=RUNNABLE, group=TGRP-TestIndexingSequenceNumbers]
[junit4] > Caused by: org.apache.lucene.index.MergePolicy$MergeException: java.lang.IllegalStateException: this writer hit an unrecoverable error; cannot merge
[junit4] > at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:704)
[junit4] > at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)
[junit4] > Caused by: java.lang.IllegalStateException: this writer hit an unrecoverable error; cannot merge
[junit4] > at org.apache.lucene.index.IndexWriter._mergeInit(IndexWriter.java:4240)
[junit4] > at org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:4220)
[junit4] > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4072
[...truncated too long message...]
d1.replica_t1, service url = null, agent id = null] for registry solr.core.deleteByCount.shard1.replica_t1 / com.codahale.metrics.MetricRegistry@1591948b
[junit4] 2> 4109067 INFO (coreCloseExecutor-7731-thread-2) [n:127.0.0.1:40378_solr c:deleteByCount s:shard1 r:core_node3 x:deleteByCount_shard1_replica_t1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.deleteByCount.shard1.leader, tag=1395551b
[junit4] 2> 4109075 INFO (jetty-closer-6306-thread-2) [ ] o.a.s.c.Overseer Overseer (id=73731937863729158-127.0.0.1:40378_solr-n_0000000000) closing
[junit4] 2> 4109075 INFO (OverseerStateUpdate-73731937863729158-127.0.0.1:40378_solr-n_0000000000) [n:127.0.0.1:40378_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:40378_solr
[junit4] 2> 4109076 WARN (OverseerAutoScalingTriggerThread-73731937863729158-127.0.0.1:40378_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 4109083 INFO (jetty-closer-6306-thread-4) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@7403d0de{/solr,null,UNAVAILABLE}
[junit4] 2> 4109084 INFO (jetty-closer-6306-thread-4) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 4109091 INFO (jetty-closer-6306-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@119b8e0c{/solr,null,UNAVAILABLE}
[junit4] 2> 4109096 INFO (jetty-closer-6306-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4d762d5a{/solr,null,UNAVAILABLE}
[junit4] 2> 4109096 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x105f2ce8cf8001d, likely client has closed socket
[junit4] 2> 4109096 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x105f2ce8cf80006, likely client has closed socket
[junit4] 2> 4109105 INFO (jetty-closer-6306-thread-1) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 4109105 INFO (jetty-closer-6306-thread-2) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 4109109 ERROR (SUITE-DeleteReplicaTest-seed#[1C427C2D2F9B9AF5]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 4109109 INFO (SUITE-DeleteReplicaTest-seed#[1C427C2D2F9B9AF5]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:43141 43141
[junit4] 2> 4109299 INFO (Thread-4678) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:43141 43141
[junit4] 2> 4109300 WARN (Thread-4678) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 15 /solr/aliases.json
[junit4] 2> 10 /solr/collections/raceDeleteReplica_true/terms/shard1
[junit4] 2> 10 /solr/configs/conf
[junit4] 2> 9 /solr/collections/deleteReplicaOnIndexing/terms/shard1
[junit4] 2> 9 /solr/clusterprops.json
[junit4] 2> 7 /solr/security.json
[junit4] 2> 7 /solr/collections/deletereplica_test/terms/shard1
[junit4] 2> 6 /solr/collections/deleteByCount/terms/shard1
[junit4] 2> 6 /solr/collections/raceDeleteReplica_false/terms/shard1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 16 /solr/collections/raceDeleteReplica_true/state.json
[junit4] 2> 15 /solr/clusterprops.json
[junit4] 2> 15 /solr/clusterstate.json
[junit4] 2> 4 /solr/collections/deleteByCount/state.json
[junit4] 2> 4 /solr/overseer_elect/election/73731937863729158-127.0.0.1:40378_solr-n_0000000000
[junit4] 2> 3 /solr/collections/deleteReplicaOnIndexing/state.json
[junit4] 2> 3 /solr/collections/raceDeleteReplica_false/state.json
[junit4] 2> 2 /solr/collections/deletereplica_test/state.json
[junit4] 2> 2 /solr/overseer_elect/election/73731937863729173-127.0.0.1:39144_solr-n_0000000004
[junit4] 2> 2 /solr/collections/raceDeleteReplica_false/leader_elect/shard1/election/73731937863729165-core_node4-n_0000000000
[junit4] 2> 2 /solr/collections/raceDeleteReplica_true/leader_elect/shard1/election/73731937863729160-core_node3-n_0000000000
[junit4] 2> 2 /solr/overseer_elect/election/73731937863729165-127.0.0.1:46158_solr-n_0000000002
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 15 /solr/live_nodes
[junit4] 2> 15 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.DeleteReplicaTest_1C427C2D2F9B9AF5-001
[junit4] 2> Nov 26, 2018 5:46:39 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene80): {id=PostingsFormat(name=Asserting)}, docValues:{}, maxPointsInLeafNode=1392, maxMBSortInHeap=6.6148065966552165, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@17c1f802), locale=hi-IN, timezone=Europe/Saratov
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_191 (64-bit)/cpus=4,threads=1,free=105365904,total=524812288
[junit4] 2> NOTE: All tests run in this JVM: [HLLUtilTest, CdcrBootstrapTest, ConfigSetsAPITest, TestManagedStopFilterFactory, RegexBytesRefFilterTest, ResourceLoaderTest, ResponseHeaderTest, ZookeeperStatusHandlerTest, TestSSLRandomization, TestRestoreCore, MetricTriggerIntegrationTest, DateFieldTest, TestLMDirichletSimilarityFactory, SuggesterWFSTTest, SliceStateTest, HdfsRecoverLeaseTest, SchemaApiFailureTest, TestSmileRequest, TestExceedMaxTermLength, HDFSCollectionsAPITest, SecurityConfHandlerTest, QueryParsingTest, SpellCheckCollatorTest, LargeFieldTest, TestPartialUpdateDeduplication, TestExportWriter, TestSearchPerf, ZkCLITest, CreateRoutedAliasTest, LeaderFailoverAfterPartitionTest, TestSolrQueryParser, TestSimpleTextCodec, StatelessScriptUpdateProcessorFactoryTest, DefaultValueUpdateProcessorTest, TestCSVResponseWriter, RollingRestartTest, TestDistributedGrouping, TestPayloadCheckQParserPlugin, TestUseDocValuesAsStored2, TestEmbeddedSolrServerSchemaAPI, HighlighterConfigTest, TestPullReplica, TestRecoveryHdfs, DirectSolrSpellCheckerTest, ScriptEngineTest, TestReversedWildcardFilterFactory, SimpleMLTQParserTest, AnalysisErrorHandlingTest, TestDownShardTolerantSearch, BasicAuthIntegrationTest, TestStressLucene, PeerSyncTest, TestSimpleTrackingShardHandler, TestNumericTerms64, AtomicUpdateProcessorFactoryTest, MigrateRouteKeyTest, ChangedSchemaMergeTest, NodeMarkersRegistrationTest, CoreAdminRequestStatusTest, TestMaxScoreQueryParser, TestSolrConfigHandlerCloud, TestMergePolicyConfig, TestInPlaceUpdatesStandalone, TestLeaderElectionWithEmptyReplica, MultiThreadedOCPTest, SolrCloudReportersTest, TestJsonFacetsWithNestedObjects, DebugComponentTest, OverseerTest, LeaderVoteWaitTimeoutTest, SolrTestCaseJ4Test, TestFacetMethods, DateMathParserTest, AutoScalingHandlerTest, LeaderElectionContextKeyTest, TestManagedSchema, AnalyticsQueryTest, AssignBackwardCompatibilityTest, NumberUtilsTest, TestBulkSchemaConcurrent, TestAddFieldRealTimeGet, SimplePostToolTest, TestCloudNestedDocsSort, SolrCoreMetricManagerTest, HdfsSyncSliceTest, TestRestManager, TestWriterPerf, SearchRateTriggerTest, TestInitParams, TestLegacyNumericUtils, TestCloudInspectUtil, TestSchemalessBufferedUpdates, TestCharFilters, TestStressInPlaceUpdates, BadCopyFieldTest, TestUpdate, TestLuceneIndexBackCompat, TestTrie, TestWithCollection, HdfsDirectoryFactoryTest, TestDynamicURP, RecoveryZkTest, ByteBuffersDirectoryFactoryTest, OpenExchangeRatesOrgProviderTest, TestTestInjection, TestRandomCollapseQParserPlugin, URLClassifyProcessorTest, TestSolrCloudWithKerberosAlt, SpellingQueryConverterTest, ZkNodePropsTest, SolrCmdDistributorTest, TestSimDistribStateManager, BasicDistributedZkTest, TestMiniSolrCloudClusterSSL, SolrMetricsIntegrationTest, DistributedTermsComponentTest, TestMaxTokenLenTokenizer, TestLFUCache, TestFuzzyAnalyzedSuggestions, TestDocumentBuilder, TriggerSetPropertiesIntegrationTest, SolrIndexSplitterTest, UninvertDocValuesMergePolicyTest, DocValuesNotIndexedTest, SolrXmlInZkTest, CoreSorterTest, TestCollationField, TestFieldCacheSort, BigEndianAscendingWordDeserializerTest, PingRequestHandlerTest, TestBlendedInfixSuggestions, HdfsChaosMonkeyNothingIsSafeTest, OverriddenZkACLAndCredentialsProvidersTest, FieldAnalysisRequestHandlerTest, UpdateRequestProcessorFactoryTest, TestJoin, TestRandomDVFaceting, TestSolrCoreProperties, TestTolerantSearch, TestWordDelimiterFilterFactory, TokenizerChainTest, ActionThrottleTest, AddReplicaTest, AliasIntegrationTest, ClusterStateTest, ClusterStateUpdateTest, CollectionPropsTest, CollectionStateFormat2Test, ConnectionManagerTest, CreateCollectionCleanupTest, DeleteReplicaTest]
[junit4] Completed [464/836 (3!)] on J2 in 141.89s, 7 tests, 1 failure, 2 errors, 1 skipped <<< FAILURES!
[...truncated 10332 lines...]
BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:651: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:585: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid5541.hprof
Total time: 600 minutes 10 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1705 - Still
unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1705/
3 tests failed.
FAILED: org.apache.solr.cloud.RestartWhileUpdatingTest.test
Error Message:
There are still nodes recoverying - waited for 320 seconds
Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 320 seconds
at __randomizedtesting.SeedInfo.seed([64E8436FE8A66828:ECBC7CB5465A05D0]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:185)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:920)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForThingsToLevelOut(AbstractFullDistribZkTestBase.java:1477)
at org.apache.solr.cloud.RestartWhileUpdatingTest.test(RestartWhileUpdatingTest.java:145)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1010)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.RestartWhileUpdatingTest
Error Message:
7 threads leaked from SUITE scope at org.apache.solr.cloud.RestartWhileUpdatingTest: 1) Thread[id=24271, name=searcherExecutor-8049-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=24464, name=searcherExecutor-8111-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=24499, name=searcherExecutor-8125-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=24228, name=searcherExecutor-8035-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=24429, name=searcherExecutor-8097-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=24308, name=searcherExecutor-8063-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=24352, name=searcherExecutor-8077-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 7 threads leaked from SUITE scope at org.apache.solr.cloud.RestartWhileUpdatingTest:
1) Thread[id=24271, name=searcherExecutor-8049-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=24464, name=searcherExecutor-8111-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=24499, name=searcherExecutor-8125-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=24228, name=searcherExecutor-8035-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=24429, name=searcherExecutor-8097-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=24308, name=searcherExecutor-8063-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
7) Thread[id=24352, name=searcherExecutor-8077-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([64E8436FE8A66828]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.RestartWhileUpdatingTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=24271, name=searcherExecutor-8049-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=24464, name=searcherExecutor-8111-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=24499, name=searcherExecutor-8125-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=24228, name=searcherExecutor-8035-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=24429, name=searcherExecutor-8097-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=24308, name=searcherExecutor-8063-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=24352, name=searcherExecutor-8077-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=24271, name=searcherExecutor-8049-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=24464, name=searcherExecutor-8111-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=24499, name=searcherExecutor-8125-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=24228, name=searcherExecutor-8035-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=24429, name=searcherExecutor-8097-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=24308, name=searcherExecutor-8063-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
7) Thread[id=24352, name=searcherExecutor-8077-thread-1, state=WAITING, group=TGRP-RestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([64E8436FE8A66828]:0)
Build Log:
[...truncated 14738 lines...]
[junit4] Suite: org.apache.solr.cloud.RestartWhileUpdatingTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_64E8436FE8A66828-001/init-core-data-001
[junit4] 2> 2389544 WARN (SUITE-RestartWhileUpdatingTest-seed#[64E8436FE8A66828]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
[junit4] 2> 2389544 INFO (SUITE-RestartWhileUpdatingTest-seed#[64E8436FE8A66828]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 2389547 INFO (SUITE-RestartWhileUpdatingTest-seed#[64E8436FE8A66828]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 2389547 INFO (SUITE-RestartWhileUpdatingTest-seed#[64E8436FE8A66828]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 2389547 INFO (SUITE-RestartWhileUpdatingTest-seed#[64E8436FE8A66828]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_xm/
[junit4] 2> 2389549 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2389549 INFO (Thread-8918) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2389549 INFO (Thread-8918) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2389552 ERROR (Thread-8918) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2389649 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.ZkTestServer start zk server on port:42064
[junit4] 2> 2389655 INFO (zkConnectionManagerCallback-5501-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2389664 INFO (zkConnectionManagerCallback-5503-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2389671 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 2389674 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 2389678 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2389680 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 2389682 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 2389683 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 2389685 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 2389687 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 2389688 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 2389690 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 2389692 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 2389694 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 2389891 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_191-b12
[junit4] 2> 2389892 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2389892 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2389892 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 2389895 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1696315f{/_xm,null,AVAILABLE}
[junit4] 2> 2389895 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2061b192{HTTP/1.1,[http/1.1]}{127.0.0.1:42557}
[junit4] 2> 2389895 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.e.j.s.Server Started @2389965ms
[junit4] 2> 2389895 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_64E8436FE8A66828-001/tempDir-001/control/data, hostContext=/_xm, hostPort=42557, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_64E8436FE8A66828-001/control-001/cores}
[junit4] 2> 2389896 ERROR (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2389896 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2389896 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 2389896 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2389896 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2389896 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-11-25T00:13:41.274Z
[junit4] 2> 2389902 INFO (zkConnectionManagerCallback-5505-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2389904 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2389904 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_64E8436FE8A66828-001/control-001/solr.xml
[junit4] 2> 2389911 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2389911 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2389915 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7105bef0, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2390078 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42064/solr
[junit4] 2> 2390080 INFO (zkConnectionManagerCallback-5509-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2390081 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100fcb338af0003, likely client has closed socket
[junit4] 2> 2390082 INFO (zkConnectionManagerCallback-5511-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2390363 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [n:127.0.0.1:42557__xm ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42557__xm
[junit4] 2> 2390369 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [n:127.0.0.1:42557__xm ] o.a.s.c.Overseer Overseer (id=72335440718266372-127.0.0.1:42557__xm-n_0000000000) starting
[junit4] 2> 2390430 INFO (zkConnectionManagerCallback-5518-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2390434 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [n:127.0.0.1:42557__xm ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42064/solr ready
[junit4] 2> 2390450 INFO (OverseerStateUpdate-72335440718266372-127.0.0.1:42557__xm-n_0000000000) [n:127.0.0.1:42557__xm ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:42557__xm
[junit4] 2> 2390454 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [n:127.0.0.1:42557__xm ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42557__xm
[junit4] 2> 2390468 INFO (zkCallback-5517-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2390472 INFO (OverseerStateUpdate-72335440718266372-127.0.0.1:42557__xm-n_0000000000) [n:127.0.0.1:42557__xm ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2390528 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [n:127.0.0.1:42557__xm ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 2390570 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [n:127.0.0.1:42557__xm ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7105bef0
[junit4] 2> 2390585 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [n:127.0.0.1:42557__xm ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7105bef0
[junit4] 2> 2390585 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [n:127.0.0.1:42557__xm ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7105bef0
[junit4] 2> 2390587 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [n:127.0.0.1:42557__xm ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_64E8436FE8A66828-001/control-001/cores
[junit4] 2> 2390623 INFO (zkConnectionManagerCallback-5524-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2390625 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2390626 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42064/solr ready
[junit4] 2> 2390628 INFO (qtp925492170-24188) [n:127.0.0.1:42557__xm ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:42557__xm&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2390635 INFO (OverseerThreadFactory-8030-thread-1-processing-n:127.0.0.1:42557__xm) [n:127.0.0.1:42557__xm ] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 2390643 DEBUG (OverseerStateUpdate-72335440718266372-127.0.0.1:42557__xm-n_0000000000) [n:127.0.0.1:42557__xm ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "name":"control_collection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"1",
[junit4] 2> "createNodeSet":"127.0.0.1:42557__xm",
[junit4] 2> "nrtReplicas":"1",
[junit4] 2> "stateFormat":"2",
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "operation":"create"} current state version: 0
[junit4] 2> 2390747 INFO (qtp925492170-24184) [n:127.0.0.1:42557__xm ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2390753 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm x:control_collection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2390753 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm x:control_collection_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2390757 DEBUG (OverseerStateUpdate-72335440718266372-127.0.0.1:42557__xm-n_0000000000) [n:127.0.0.1:42557__xm ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":"control_collection_shard1_replica_n1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:42557/_xm",
[junit4] 2> "node_name":"127.0.0.1:42557__xm",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 2391777 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2391796 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 2391914 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 2391934 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 2391935 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7105bef0
[junit4] 2> 2391935 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_64E8436FE8A66828-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_64E8436FE8A66828-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 2391939 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=30, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=16.875, floorSegmentMB=1.919921875, forceMergeDeletesPctAllowed=1.6235389740349304, segmentsPerTier=24.0, maxCFSSegmentSizeMB=0.5341796875, noCFSRatio=1.0, deletesPctAllowed=25.991601486455025
[junit4] 2> 2391947 WARN (qtp925492170-24183) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2392014 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2392015 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2392016 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommitted for 30000ms;
[junit4] 2> 2392016 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: if uncommitted for 3000ms;
[junit4] 2> 2392017 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@4705ca78
[junit4] 2> 2392018 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2dbbd6a3[control_collection_shard1_replica_n1] main]
[junit4] 2> 2392019 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2392020 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2392021 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2392021 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1618062683300429824
[junit4] 2> 2392022 INFO (searcherExecutor-8035-thread-1-processing-n:127.0.0.1:42557__xm x:control_collection_shard1_replica_n1 c:control_collection s:shard1) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@2dbbd6a3[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2392028 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/control_collection/terms/shard1 to Terms{values={core_node2=0}, version=0}
[junit4] 2> 2392033 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2392033 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2392033 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:42557/_xm/control_collection_shard1_replica_n1/
[junit4] 2> 2392033 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2392034 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:42557/_xm/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 2392039 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:42557/_xm/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 2392039 DEBUG (OverseerStateUpdate-72335440718266372-127.0.0.1:42557__xm-n_0000000000) [n:127.0.0.1:42557__xm ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "base_url":"http://127.0.0.1:42557/_xm",
[junit4] 2> "core":"control_collection_shard1_replica_n1",
[junit4] 2> "state":"active"} current state version: 0
[junit4] 2> 2392142 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2392144 DEBUG (OverseerStateUpdate-72335440718266372-127.0.0.1:42557__xm-n_0000000000) [n:127.0.0.1:42557__xm ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":"control_collection_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:42557/_xm",
[junit4] 2> "node_name":"127.0.0.1:42557__xm",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 2392145 INFO (qtp925492170-24183) [n:127.0.0.1:42557__xm c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1392
[junit4] 2> 2392148 INFO (qtp925492170-24188) [n:127.0.0.1:42557__xm ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 2392245 INFO (zkCallback-5510-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2392633 INFO (OverseerCollectionConfigSetProcessor-72335440718266372-127.0.0.1:42557__xm-n_0000000000) [n:127.0.0.1:42557__xm ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2393148 INFO (qtp925492170-24188) [n:127.0.0.1:42557__xm ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:42557__xm&wt=javabin&version=2} status=0 QTime=2520
[junit4] 2> 2393154 INFO (zkConnectionManagerCallback-5529-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2393155 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2393157 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42064/solr ready
[junit4] 2> 2393157 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 2393158 INFO (qtp925492170-24187) [n:127.0.0.1:42557__xm ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2393165 INFO (OverseerThreadFactory-8030-thread-2-processing-n:127.0.0.1:42557__xm) [n:127.0.0.1:42557__xm ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 2393169 DEBUG (OverseerStateUpdate-72335440718266372-127.0.0.1:42557__xm-n_0000000000) [n:127.0.0.1:42557__xm ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "name":"collection1",
[junit4] 2> "fromApi":"true",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"1",
[junit4] 2> "createNodeSet":"",
[junit4] 2> "stateFormat":"1",
[junit4] 2> "nrtReplicas":"1",
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "operation":"create"} current state version: 0
[junit4] 2> 2393369 WARN (OverseerThreadFactory-8030-thread-2-processing-n:127.0.0.1:42557__xm) [n:127.0.0.1:42557__xm ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 2393372 INFO (qtp925492170-24187) [n:127.0.0.1:42557__xm ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 2393372 INFO (qtp925492170-24187) [n:127.0.0.1:42557__xm ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=213
[junit4] 2> 2393555 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_64E8436FE8A66828-001/shard-1-001 of type NRT
[junit4] 2> 2393556 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_191-b12
[junit4] 2> 2393557 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2393557 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2393557 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 2393558 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7441b042{/_xm,null,AVAILABLE}
[junit4] 2> 2393558 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5a9bdee5{HTTP/1.1,[http/1.1]}{127.0.0.1:36433}
[junit4] 2> 2393558 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.e.j.s.Server Started @2393628ms
[junit4] 2> 2393558 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_64E8436FE8A66828-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/_xm, hostPort=36433, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_64E8436FE8A66828-001/shard-1-001/cores}
[junit4] 2> 2393559 ERROR (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2393559 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2393559 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 2393559 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2393559 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2393559 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-11-25T00:13:44.937Z
[junit4] 2> 2393562 INFO (zkConnectionManagerCallback-5531-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2393563 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2393563 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_64E8436FE8A66828-001/shard-1-001/solr.xml
[junit4] 2> 2393569 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2393569 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2393571 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7105bef0, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2393653 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42064/solr
[junit4] 2> 2393654 INFO (zkConnectionManagerCallback-5535-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2393656 INFO (zkConnectionManagerCallback-5537-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2393666 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2393673 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2393673 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36433__xm
[junit4] 2> 2393674 INFO (zkCallback-5517-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2393674 INFO (zkCallback-5528-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2393677 INFO (zkCallback-5510-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2393677 INFO (zkCallback-5536-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2393677 DEBUG (OverseerStateUpdate-72335440718266372-127.0.0.1:42557__xm-n_0000000000) [n:127.0.0.1:42557__xm ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:36433__xm"} current state version: 1
[junit4] 2> 2393705 INFO (zkConnectionManagerCallback-5544-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2393707 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2393709 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42064/solr ready
[junit4] 2> 2393710 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 2393744 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7105bef0
[junit4] 2> 2393762 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7105bef0
[junit4] 2> 2393763 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7105bef0
[junit4] 2> 2393765 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_64E8436FE8A66828-001/shard-1-001/cores
[junit4] 2> 2393808 INFO (qtp370342008-24243) [n:127.0.0.1:36433__xm ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:36433__xm&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2393812 INFO (OverseerCollectionConfigSetProcessor-72335440718266372-127.0.0.1:42557__xm-n_0000000000) [n:127.0.0.1:42557__xm ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2393815 INFO (qtp370342008-24238) [n:127.0.0.1:36433__xm ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2393817 INFO (qtp925492170-24184) [n:127.0.0.1:42557__xm ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.control_collection.shard1.replica_n1:INDEX.sizeInBytes} status=0 QTime=1
[junit4] 2> 2393819 INFO (qtp925492170-24187) [n:127.0.0.1:42557__xm ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2393819 INFO (OverseerThreadFactory-8030-thread-3-processing-n:127.0.0.1:42557__xm) [n:127.0.0.1:42557__xm c:collection1 s:shard1 ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:36433__xm for creating new replica of shard shard1 for collection collection1
[junit4] 2> 2393822 INFO (qtp370342008-24239) [n:127.0.0.1:36433__xm x:collection1_shard1_replica_n21] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n21&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2393825 DEBUG (OverseerStateUpdate-72335440718266372-127.0.0.1:42557__xm-n_0000000000) [n:127.0.0.1:42557__xm ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1_shard1_replica_n21",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:36433/_xm",
[junit4] 2> "node_name":"127.0.0.1:36433__xm",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 1
[junit4] 2> 2394846 INFO (qtp370342008-24239) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2394880 INFO (qtp370342008-24239) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.IndexSchema [collection1_shard1_replica_n21] Schema name=test
[junit4] 2> 2394993 INFO (qtp370342008-24239) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 2395013 INFO (qtp370342008-24239) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n21' using configuration from collection collection1, trusted=true
[junit4] 2> 2395014 INFO (qtp370342008-24239) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n21' (registry 'solr.core.collection1.shard1.replica_n21') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7105bef0
[junit4] 2> 2395014 INFO (qtp370342008-24239) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrCore [[collection1_shard1_replica_n21] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_64E8436FE8A66828-001/shard-1-001/cores/collection1_shard1_replica_n21], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_64E8436FE8A66828-001/shard-1-001/cores/collection1_shard1_replica_n21/data/]
[junit4] 2> 2395029 INFO (qtp370342008-24239) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=30, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=16.875, floorSegmentMB=1.919921875, forceMergeDeletesPctAllowed=1.6235389740349304, segmentsPerTier=24.0, maxCFSSegmentSizeMB=0.5341796875, noCFSRatio=1.0, deletesPctAllowed=25.991601486455025
[junit4] 2> 2395039 WARN (qtp370342008-24239) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2395106 INFO (qtp370342008-24239) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2395107 INFO (qtp370342008-24239) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2395108 INFO (qtp370342008-24239) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.CommitTracker Hard AutoCommit: if uncommitted for 30000ms;
[junit4] 2> 2395108 INFO (qtp370342008-24239) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.CommitTracker Soft AutoCommit: if uncommitted for 3000ms;
[junit4] 2> 2395110 INFO (qtp370342008-24239) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@25fa6d77
[junit4] 2> 2395110 INFO (qtp370342008-24239) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.SolrIndexSearcher Opening [Searcher@329684db[collection1_shard1_replica_n21] main]
[junit4] 2> 2395112 INFO (qtp370342008-24239) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2395112 INFO (qtp370342008-24239) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2395113 INFO (qtp370342008-24239) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2395113 INFO (qtp370342008-24239) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1618062686542626816
[junit4] 2> 2395115 INFO (searcherExecutor-8049-thread-1-processing-n:127.0.0.1:36433__xm x:collection1_shard1_replica_n21 c:collection1 s:shard1) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrCore [collection1_shard1_replica_n21] Registered new searcher Searcher@329684db[collection1_shard1_replica_n21] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2395120 INFO (qtp370342008-24239) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node22=0}, version=0}
[junit4] 2> 2395124 INFO (qtp370342008-24239) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2395124 INFO (qtp370342008-24239) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2395124 INFO (qtp370342008-24239) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36433/_xm/collection1_shard1_replica_n21/
[junit4] 2> 2395124 INFO (qtp370342008-24239) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2395124 INFO (qtp370342008-24239) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SyncStrategy http://127.0.0.1:36433/_xm/collection1_shard1_replica_n21/ has no replicas
[junit4] 2> 2395128 INFO (qtp370342008-24239) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36433/_xm/collection1_shard1_replica_n21/ shard1
[junit4] 2> 2395129 DEBUG (OverseerStateUpdate-72335440718266372-127.0.0.1:42557__xm-n_0000000000) [n:127.0.0.1:42557__xm ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "base_url":"http://127.0.0.1:36433/_xm",
[junit4] 2> "core":"collection1_shard1_replica_n21",
[junit4] 2> "state":"active"} current state version: 2
[junit4] 2> 2395279 INFO (qtp370342008-24239) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2395281 DEBUG (OverseerStateUpdate-72335440718266372-127.0.0.1:42557__xm-n_0000000000) [n:127.0.0.1:42557__xm ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1_shard1_replica_n21",
[junit4] 2> "core_node_name":"core_node22",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:36433/_xm",
[junit4] 2> "node_name":"127.0.0.1:36433__xm",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 3
[junit4] 2> 2395282 INFO (qtp370342008-24239) [n:127.0.0.1:36433__xm c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n21&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1460
[junit4] 2> 2395289 INFO (qtp370342008-24243) [n:127.0.0.1:36433__xm c:collection1 ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:36433__xm&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=1480
[junit4] 2> 2395495 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_64E8436FE8A66828-001/shard-2-001 of type NRT
[junit4] 2> 2395495 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_191-b12
[junit4] 2> 2395497 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2395497 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2395497 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 2395497 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4e881265{/_xm,null,AVAILABLE}
[junit4] 2> 2395498 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6b849516{HTTP/1.1,[http/1.1]}{127.0.0.1:42892}
[junit4] 2> 2395498 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.e.j.s.Server Started @2395568ms
[junit4] 2> 2395498 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_64E8436FE8A66828-001/tempDir-001/jetty2, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/_xm, hostPort=42892, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_64E8436FE8A66828-001/shard-2-001/cores}
[junit4] 2> 2395499 ERROR (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2395500 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2395500 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 2395500 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2395500 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2395500 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-11-25T00:13:46.878Z
[junit4] 2> 2395502 INFO (zkConnectionManagerCallback-5547-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2395503 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2395504 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_64E8436FE8A66828-001/shard-2-001/solr.xml
[junit4] 2> 2395509 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2395509 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2395511 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7105bef0, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2395754 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42064/solr
[junit4] 2> 2395756 INFO (zkConnectionManagerCallback-5551-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2395759 INFO (zkConnectionManagerCallback-5553-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2395769 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2395775 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2395775 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42892__xm
[junit4] 2> 2395776 DEBUG (OverseerStateUpdate-72335440718266372-127.0.0.1:42557__xm-n_0000000000) [n:127.0.0.1:42557__xm ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:42892__xm"} current state version: 4
[junit4] 2> 2395777 INFO (zkCallback-5536-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2395777 INFO (zkCallback-5528-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2395777 INFO (zkCallback-5510-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2395778 INFO (zkCallback-5543-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2395778 INFO (zkCallback-5517-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2395780 INFO (zkCallback-5552-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2395806 INFO (zkConnectionManagerCallback-5560-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2395807 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2395809 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42064/solr ready
[junit4] 2> 2395809 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 2395814 INFO (OverseerCollectionConfigSetProcessor-72335440718266372-127.0.0.1:42557__xm-n_0000000000) [n:127.0.0.1:42557__xm ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2395840 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7105bef0
[junit4] 2> 2395858 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7105bef0
[junit4] 2> 2395859 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7105bef0
[junit4] 2> 2395861 INFO (TEST-RestartWhileUpdatingTest.test-seed#[64E8436FE8A66828]) [ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.RestartWhileUpdatingTest_64E8436FE8A66828-001/shard-2-001/cores
[junit4] 2> 2395916 INFO (qtp370342008-24238) [n:127.0.0.1:36433__xm ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:42892__xm&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2395923 INFO (qtp370342008-24242) [n:127.0.0.1:36433__xm ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.collection1.shard1.replica_n21:INDEX.sizeInBytes} status=0 QTime=1
[junit4] 2> 2395924 INFO (qtp370342008-24245) [n:127.0.0.1:36433__xm ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2395925 INFO (qtp925492170-24184) [n:127.0.0.1:42557__xm ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.control_collection.shard1.replica_n1:INDEX.sizeInBytes} status=0 QTime=0
[junit4] 2> 2395926 INFO (qtp925492170-24187) [n:127.0.0.1:42557__xm ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2395928 INFO (qtp876885733-24280) [n:127.0.0.1:42892__xm ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2395929 INFO (qtp370342008-24242) [n:127.0.0.1:36433__xm ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.collection1.shard1.replica_n21:INDEX.sizeInBytes} status=0 QTime=0
[junit4] 2> 2395930 INFO (OverseerThreadFactory-8030-thread-4-processing-n:127.0.0.1:42557__xm) [n:127.0.0.1:42557__xm c:collection1 s:shard1 ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:42892__xm for creating new replica of shard shard1 for collection collection1
[junit4] 2> 2395933 INFO (qtp876885733-24275) [n:127.0.0.1:42892__xm x:collection1_shard1_replica_n23] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n23&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2395935 DEBUG (OverseerStateUpdate-72335440718266372-127.0.0.1:42557__xm-n_0000000000) [n:127.0.0.1:42557__xm ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":"collection1_shard1_replica_n23",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:42892/_xm",
[junit4] 2> "node_name":"127.0.0.1:42892__xm",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 4
[junit4] 2> 2396955 INFO (qtp876885733-24275) [n:127.0.0.1:42892__xm c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2396978 INFO (qtp876885733-24275) [n:127.0.0.1:42892__xm c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.s.IndexSchema [collection1_shard1_replica_n23] Schema name=test
[junit4] 2> 2397116 INFO (qtp876885733-24275) [n:127.0.0.1:42892__xm c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 2397138 INFO (qtp876885733-24275) [n:127.0.0.1:42892__xm c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.CoreCo
[...truncated too long message...]
ve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/null958687154
[copy] Copying 240 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/null958687154
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/null958687154
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 36ms :: artifacts dl 2ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 280 minutes 56 seconds
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
at hudson.FilePath$34.hasMatch(FilePath.java:2678)
at hudson.FilePath$34.invoke(FilePath.java:2557)
at hudson.FilePath$34.invoke(FilePath.java:2547)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:955)
at hudson.FilePath.act(FilePath.java:1036)
at hudson.FilePath.act(FilePath.java:1025)
at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1819)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused: hudson.FilePath$TunneledInterruptedException
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2920)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
at hudson.FilePath.act(FilePath.java:1038)
at hudson.FilePath.act(FilePath.java:1025)
at hudson.FilePath.validateAntFileMask(FilePath.java:2547)
at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1819)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1704 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1704/
4 tests failed.
FAILED: org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica
Error Message:
Expected new active leader null Live Nodes: [127.0.0.1:36001_solr, 127.0.0.1:39242_solr, 127.0.0.1:42552_solr] Last available state: DocCollection(raceDeleteReplica_false//collections/raceDeleteReplica_false/state.json/13)={ "pullReplicas":"0", "replicationFactor":"2", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{ "core_node4":{ "core":"raceDeleteReplica_false_shard1_replica_n3", "base_url":"http://127.0.0.1:43824/solr", "node_name":"127.0.0.1:43824_solr", "state":"down", "type":"NRT", "force_set_state":"false", "leader":"true"}, "core_node6":{ "core":"raceDeleteReplica_false_shard1_replica_n5", "base_url":"http://127.0.0.1:43824/solr", "node_name":"127.0.0.1:43824_solr", "state":"down", "type":"NRT", "force_set_state":"false"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0"}
Stack Trace:
java.lang.AssertionError: Expected new active leader
null
Live Nodes: [127.0.0.1:36001_solr, 127.0.0.1:39242_solr, 127.0.0.1:42552_solr]
Last available state: DocCollection(raceDeleteReplica_false//collections/raceDeleteReplica_false/state.json/13)={
"pullReplicas":"0",
"replicationFactor":"2",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{
"core_node4":{
"core":"raceDeleteReplica_false_shard1_replica_n3",
"base_url":"http://127.0.0.1:43824/solr",
"node_name":"127.0.0.1:43824_solr",
"state":"down",
"type":"NRT",
"force_set_state":"false",
"leader":"true"},
"core_node6":{
"core":"raceDeleteReplica_false_shard1_replica_n5",
"base_url":"http://127.0.0.1:43824/solr",
"node_name":"127.0.0.1:43824_solr",
"state":"down",
"type":"NRT",
"force_set_state":"false"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0"}
at __randomizedtesting.SeedInfo.seed([30E77759D5165AE3:5AF11689BDE41029]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
at org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica(DeleteReplicaTest.java:328)
at org.apache.solr.cloud.DeleteReplicaTest.raceConditionOnDeleteAndRegisterReplica(DeleteReplicaTest.java:224)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.DeleteReplicaTest.deleteReplicaByCountForAllShards
Error Message:
Error from server at http://127.0.0.1:42552/solr: Cannot create collection deleteByCountNew. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 3. This allows a maximum of 3 to be created. Value of numShards is 2, value of nrtReplicas is 2, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 4 shards to be created (higher than the allowed number)
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:42552/solr: Cannot create collection deleteByCountNew. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 3. This allows a maximum of 3 to be created. Value of numShards is 2, value of nrtReplicas is 2, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 4 shards to be created (higher than the allowed number)
at __randomizedtesting.SeedInfo.seed([30E77759D5165AE3:C76300D9A5707B6]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.DeleteReplicaTest.deleteReplicaByCountForAllShards(DeleteReplicaTest.java:161)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test
Error Message:
Java heap space
Stack Trace:
java.lang.OutOfMemoryError: Java heap space
FAILED: org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest.test
Error Message:
There are still nodes recoverying - waited for 320 seconds
Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 320 seconds
at __randomizedtesting.SeedInfo.seed([30E77759D5165AE3:B8B348837BEA371B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:185)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:920)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForThingsToLevelOut(AbstractFullDistribZkTestBase.java:1477)
at org.apache.solr.cloud.RestartWhileUpdatingTest.test(RestartWhileUpdatingTest.java:145)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1010)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13793 lines...]
[junit4] Suite: org.apache.solr.cloud.DeleteReplicaTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest_30E77759D5165AE3-001/init-core-data-001
[junit4] 2> 641430 WARN (SUITE-DeleteReplicaTest-seed#[30E77759D5165AE3]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=33 numCloses=33
[junit4] 2> 641439 INFO (SUITE-DeleteReplicaTest-seed#[30E77759D5165AE3]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 641441 INFO (SUITE-DeleteReplicaTest-seed#[30E77759D5165AE3]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 641441 INFO (SUITE-DeleteReplicaTest-seed#[30E77759D5165AE3]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 641442 INFO (SUITE-DeleteReplicaTest-seed#[30E77759D5165AE3]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest_30E77759D5165AE3-001/tempDir-001
[junit4] 2> 641442 INFO (SUITE-DeleteReplicaTest-seed#[30E77759D5165AE3]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 641455 INFO (Thread-1249) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 641455 INFO (Thread-1249) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 641479 ERROR (Thread-1249) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 641547 INFO (SUITE-DeleteReplicaTest-seed#[30E77759D5165AE3]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:34369
[junit4] 2> 641571 INFO (zkConnectionManagerCallback-1521-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 641576 INFO (jetty-launcher-1518-thread-1) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_191-b12
[junit4] 2> 641580 INFO (jetty-launcher-1518-thread-2) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_191-b12
[junit4] 2> 641588 INFO (jetty-launcher-1518-thread-3) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_191-b12
[junit4] 2> 641589 INFO (jetty-launcher-1518-thread-4) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_191-b12
[junit4] 2> 641655 INFO (jetty-launcher-1518-thread-4) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 641655 INFO (jetty-launcher-1518-thread-4) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 641655 INFO (jetty-launcher-1518-thread-4) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 641655 INFO (jetty-launcher-1518-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@156a64d6{/solr,null,AVAILABLE}
[junit4] 2> 641658 INFO (jetty-launcher-1518-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 641659 INFO (jetty-launcher-1518-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 641659 INFO (jetty-launcher-1518-thread-3) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 641679 INFO (jetty-launcher-1518-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 641715 INFO (jetty-launcher-1518-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 641715 INFO (jetty-launcher-1518-thread-1) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 641710 INFO (jetty-launcher-1518-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 641715 INFO (jetty-launcher-1518-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 641715 INFO (jetty-launcher-1518-thread-2) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 641710 INFO (jetty-launcher-1518-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@33605f55{HTTP/1.1,[http/1.1]}{127.0.0.1:42552}
[junit4] 2> 641715 INFO (jetty-launcher-1518-thread-4) [ ] o.e.j.s.Server Started @641788ms
[junit4] 2> 641715 INFO (jetty-launcher-1518-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42552}
[junit4] 2> 641716 ERROR (jetty-launcher-1518-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 641716 INFO (jetty-launcher-1518-thread-4) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 641716 INFO (jetty-launcher-1518-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 641716 INFO (jetty-launcher-1518-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 641716 INFO (jetty-launcher-1518-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 641716 INFO (jetty-launcher-1518-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-11-23T14:10:53.495Z
[junit4] 2> 641689 INFO (jetty-launcher-1518-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@35ac9aa2{/solr,null,AVAILABLE}
[junit4] 2> 641728 INFO (jetty-launcher-1518-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@26d7b567{/solr,null,AVAILABLE}
[junit4] 2> 641730 INFO (jetty-launcher-1518-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2c5a7f00{HTTP/1.1,[http/1.1]}{127.0.0.1:39242}
[junit4] 2> 641755 INFO (jetty-launcher-1518-thread-3) [ ] o.e.j.s.Server Started @641828ms
[junit4] 2> 641755 INFO (jetty-launcher-1518-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39242}
[junit4] 2> 641756 ERROR (jetty-launcher-1518-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 641756 INFO (jetty-launcher-1518-thread-3) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 641756 INFO (jetty-launcher-1518-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 641756 INFO (jetty-launcher-1518-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 641756 INFO (jetty-launcher-1518-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 641756 INFO (jetty-launcher-1518-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-11-23T14:10:53.535Z
[junit4] 2> 641751 INFO (zkConnectionManagerCallback-1523-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 641763 INFO (jetty-launcher-1518-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 641755 INFO (jetty-launcher-1518-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7db6f6c2{/solr,null,AVAILABLE}
[junit4] 2> 641764 INFO (jetty-launcher-1518-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6e34d759{HTTP/1.1,[http/1.1]}{127.0.0.1:36001}
[junit4] 2> 641765 INFO (jetty-launcher-1518-thread-1) [ ] o.e.j.s.Server Started @641838ms
[junit4] 2> 641765 INFO (jetty-launcher-1518-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36001}
[junit4] 2> 641765 INFO (jetty-launcher-1518-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@10accc0{HTTP/1.1,[http/1.1]}{127.0.0.1:43824}
[junit4] 2> 641765 INFO (jetty-launcher-1518-thread-2) [ ] o.e.j.s.Server Started @641838ms
[junit4] 2> 641765 ERROR (jetty-launcher-1518-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 641765 INFO (jetty-launcher-1518-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 641765 INFO (jetty-launcher-1518-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43824}
[junit4] 2> 641765 INFO (jetty-launcher-1518-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 641765 INFO (jetty-launcher-1518-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 641765 INFO (jetty-launcher-1518-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 641765 INFO (jetty-launcher-1518-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-11-23T14:10:53.544Z
[junit4] 2> 641765 ERROR (jetty-launcher-1518-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 641765 INFO (jetty-launcher-1518-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 641765 INFO (jetty-launcher-1518-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 641765 INFO (jetty-launcher-1518-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 641765 INFO (jetty-launcher-1518-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 641766 INFO (jetty-launcher-1518-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-11-23T14:10:53.544Z
[junit4] 2> 641775 INFO (zkConnectionManagerCallback-1525-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 641776 INFO (jetty-launcher-1518-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 641791 INFO (zkConnectionManagerCallback-1529-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 641794 INFO (zkConnectionManagerCallback-1527-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 641805 INFO (jetty-launcher-1518-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 641805 INFO (jetty-launcher-1518-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 642113 INFO (jetty-launcher-1518-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34369/solr
[junit4] 2> 642128 INFO (zkConnectionManagerCallback-1533-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 642158 INFO (zkConnectionManagerCallback-1535-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 642187 INFO (jetty-launcher-1518-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34369/solr
[junit4] 2> 642235 INFO (zkConnectionManagerCallback-1541-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 642250 INFO (zkConnectionManagerCallback-1543-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 642271 INFO (jetty-launcher-1518-thread-4) [n:127.0.0.1:42552_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42552_solr
[junit4] 2> 642274 INFO (jetty-launcher-1518-thread-4) [n:127.0.0.1:42552_solr ] o.a.s.c.Overseer Overseer (id=72327408122331144-127.0.0.1:42552_solr-n_0000000000) starting
[junit4] 2> 642289 INFO (zkConnectionManagerCallback-1550-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 642296 INFO (jetty-launcher-1518-thread-4) [n:127.0.0.1:42552_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34369/solr ready
[junit4] 2> 642298 INFO (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:42552_solr
[junit4] 2> 642314 INFO (jetty-launcher-1518-thread-4) [n:127.0.0.1:42552_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42552_solr
[junit4] 2> 642331 INFO (jetty-launcher-1518-thread-1) [n:127.0.0.1:36001_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 642339 INFO (jetty-launcher-1518-thread-1) [n:127.0.0.1:36001_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 642339 INFO (jetty-launcher-1518-thread-1) [n:127.0.0.1:36001_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36001_solr
[junit4] 2> 642374 INFO (zkCallback-1542-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 642375 INFO (zkCallback-1549-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 642395 INFO (zkCallback-1534-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 642430 INFO (jetty-launcher-1518-thread-4) [n:127.0.0.1:42552_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 642435 DEBUG (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:36001_solr"} current state version: 0
[junit4] 2> 642451 INFO (jetty-launcher-1518-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34369/solr
[junit4] 2> 642472 INFO (zkConnectionManagerCallback-1560-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 642479 INFO (zkConnectionManagerCallback-1556-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 642485 INFO (jetty-launcher-1518-thread-1) [n:127.0.0.1:36001_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 642488 INFO (jetty-launcher-1518-thread-1) [n:127.0.0.1:36001_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34369/solr ready
[junit4] 2> 642499 INFO (jetty-launcher-1518-thread-4) [n:127.0.0.1:42552_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42552.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@578f7301
[junit4] 2> 642500 INFO (jetty-launcher-1518-thread-1) [n:127.0.0.1:36001_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 642528 INFO (zkConnectionManagerCallback-1562-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 642551 INFO (jetty-launcher-1518-thread-3) [n:127.0.0.1:39242_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 642552 INFO (jetty-launcher-1518-thread-4) [n:127.0.0.1:42552_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42552.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@578f7301
[junit4] 2> 642552 INFO (jetty-launcher-1518-thread-4) [n:127.0.0.1:42552_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42552.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@578f7301
[junit4] 2> 642558 INFO (jetty-launcher-1518-thread-3) [n:127.0.0.1:39242_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 642558 INFO (jetty-launcher-1518-thread-3) [n:127.0.0.1:39242_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39242_solr
[junit4] 2> 642560 DEBUG (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:39242_solr"} current state version: 0
[junit4] 2> 642560 INFO (zkCallback-1549-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 642560 INFO (zkCallback-1534-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 642560 INFO (zkCallback-1542-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 642560 INFO (jetty-launcher-1518-thread-4) [n:127.0.0.1:42552_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest_30E77759D5165AE3-001/tempDir-001/node4/.
[junit4] 2> 642579 INFO (jetty-launcher-1518-thread-1) [n:127.0.0.1:36001_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36001.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@578f7301
[junit4] 2> 642625 INFO (zkCallback-1555-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 642626 INFO (zkCallback-1561-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 642643 INFO (jetty-launcher-1518-thread-1) [n:127.0.0.1:36001_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36001.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@578f7301
[junit4] 2> 642643 INFO (jetty-launcher-1518-thread-1) [n:127.0.0.1:36001_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36001.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@578f7301
[junit4] 2> 642646 INFO (jetty-launcher-1518-thread-1) [n:127.0.0.1:36001_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest_30E77759D5165AE3-001/tempDir-001/node1/.
[junit4] 2> 642647 INFO (zkConnectionManagerCallback-1570-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 642649 INFO (jetty-launcher-1518-thread-3) [n:127.0.0.1:39242_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 642650 INFO (jetty-launcher-1518-thread-3) [n:127.0.0.1:39242_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34369/solr ready
[junit4] 2> 642676 INFO (jetty-launcher-1518-thread-3) [n:127.0.0.1:39242_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 642679 INFO (jetty-launcher-1518-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34369/solr
[junit4] 2> 642695 INFO (zkConnectionManagerCallback-1575-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 642706 INFO (jetty-launcher-1518-thread-3) [n:127.0.0.1:39242_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39242.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@578f7301
[junit4] 2> 642719 INFO (jetty-launcher-1518-thread-3) [n:127.0.0.1:39242_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39242.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@578f7301
[junit4] 2> 642719 INFO (jetty-launcher-1518-thread-3) [n:127.0.0.1:39242_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39242.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@578f7301
[junit4] 2> 642721 INFO (jetty-launcher-1518-thread-3) [n:127.0.0.1:39242_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest_30E77759D5165AE3-001/tempDir-001/node3/.
[junit4] 2> 642722 INFO (zkConnectionManagerCallback-1577-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 642733 INFO (jetty-launcher-1518-thread-2) [n:127.0.0.1:43824_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 642739 INFO (jetty-launcher-1518-thread-2) [n:127.0.0.1:43824_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 642739 INFO (jetty-launcher-1518-thread-2) [n:127.0.0.1:43824_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43824_solr
[junit4] 2> 642740 DEBUG (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:43824_solr"} current state version: 0
[junit4] 2> 642741 INFO (zkCallback-1549-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 642741 INFO (zkCallback-1534-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 642741 INFO (zkCallback-1542-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 642741 INFO (zkCallback-1555-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 642741 INFO (zkCallback-1561-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 642754 INFO (zkCallback-1569-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 642754 INFO (zkCallback-1576-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 642774 INFO (zkConnectionManagerCallback-1584-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 642778 INFO (jetty-launcher-1518-thread-2) [n:127.0.0.1:43824_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 642779 INFO (jetty-launcher-1518-thread-2) [n:127.0.0.1:43824_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34369/solr ready
[junit4] 2> 642780 INFO (jetty-launcher-1518-thread-2) [n:127.0.0.1:43824_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 642805 INFO (jetty-launcher-1518-thread-2) [n:127.0.0.1:43824_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43824.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@578f7301
[junit4] 2> 642817 INFO (jetty-launcher-1518-thread-2) [n:127.0.0.1:43824_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43824.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@578f7301
[junit4] 2> 642818 INFO (jetty-launcher-1518-thread-2) [n:127.0.0.1:43824_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43824.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@578f7301
[junit4] 2> 642819 INFO (jetty-launcher-1518-thread-2) [n:127.0.0.1:43824_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest_30E77759D5165AE3-001/tempDir-001/node2/.
[junit4] 2> 642893 INFO (zkConnectionManagerCallback-1587-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 642903 INFO (zkConnectionManagerCallback-1592-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 642913 INFO (SUITE-DeleteReplicaTest-seed#[30E77759D5165AE3]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 642916 INFO (SUITE-DeleteReplicaTest-seed#[30E77759D5165AE3]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34369/solr ready
[junit4] 2> 643018 INFO (TEST-DeleteReplicaTest.deleteReplicaAndVerifyDirectoryCleanup-seed#[30E77759D5165AE3]) [ ] o.a.s.SolrTestCaseJ4 ###Starting deleteReplicaAndVerifyDirectoryCleanup
[junit4] 2> 643025 INFO (qtp1742466670-5720) [n:127.0.0.1:42552_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf&name=deletereplica_test&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 643037 INFO (OverseerThreadFactory-2677-thread-1-processing-n:127.0.0.1:42552_solr) [n:127.0.0.1:42552_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection deletereplica_test
[junit4] 2> 643045 DEBUG (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "name":"deletereplica_test",
[junit4] 2> "fromApi":"true",
[junit4] 2> "collection.configName":"conf",
[junit4] 2> "numShards":"1",
[junit4] 2> "nrtReplicas":"2",
[junit4] 2> "stateFormat":"2",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "operation":"create"} current state version: 0
[junit4] 2> 643155 INFO (qtp1742466670-5712) [n:127.0.0.1:42552_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 643161 INFO (qtp54270777-5737) [n:127.0.0.1:43824_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 643165 INFO (qtp1833868297-5732) [n:127.0.0.1:36001_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=2
[junit4] 2> 643172 INFO (qtp639475210-5728) [n:127.0.0.1:39242_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 643194 DEBUG (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"deletereplica_test",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"deletereplica_test_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:39242/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 643194 INFO (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"deletereplica_test",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"deletereplica_test_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:39242/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 643215 DEBUG (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"deletereplica_test",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"deletereplica_test_shard1_replica_n3",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:36001/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 643215 INFO (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"deletereplica_test",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"deletereplica_test_shard1_replica_n3",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:36001/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 643433 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr x:deletereplica_test_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=deletereplica_test_shard1_replica_n1&action=CREATE&numShards=1&collection=deletereplica_test&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 643444 INFO (qtp1833868297-5709) [n:127.0.0.1:36001_solr x:deletereplica_test_shard1_replica_n3] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=deletereplica_test_shard1_replica_n3&action=CREATE&numShards=1&collection=deletereplica_test&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 643457 DEBUG (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":"deletereplica_test_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:39242/solr",
[junit4] 2> "node_name":"127.0.0.1:39242_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"deletereplica_test",
[junit4] 2> "type":"NRT",
[junit4] 2> "force_set_state":"false",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 643460 DEBUG (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "core":"deletereplica_test_shard1_replica_n3",
[junit4] 2> "core_node_name":"core_node4",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:36001/solr",
[junit4] 2> "node_name":"127.0.0.1:36001_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"deletereplica_test",
[junit4] 2> "type":"NRT",
[junit4] 2> "force_set_state":"false",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 644481 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 644486 INFO (qtp1833868297-5709) [n:127.0.0.1:36001_solr c:deletereplica_test s:shard1 r:core_node4 x:deletereplica_test_shard1_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 644518 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.s.IndexSchema [deletereplica_test_shard1_replica_n1] Schema name=minimal
[junit4] 2> 644525 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 644525 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'deletereplica_test_shard1_replica_n1' using configuration from collection deletereplica_test, trusted=true
[junit4] 2> 644526 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39242.solr.core.deletereplica_test.shard1.replica_n1' (registry 'solr.core.deletereplica_test.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@578f7301
[junit4] 2> 644526 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.c.SolrCore [[deletereplica_test_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest_30E77759D5165AE3-001/tempDir-001/node3/deletereplica_test_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest_30E77759D5165AE3-001/tempDir-001/node3/./deletereplica_test_shard1_replica_n1/data/]
[junit4] 2> 644529 INFO (qtp1833868297-5709) [n:127.0.0.1:36001_solr c:deletereplica_test s:shard1 r:core_node4 x:deletereplica_test_shard1_replica_n3] o.a.s.s.IndexSchema [deletereplica_test_shard1_replica_n3] Schema name=minimal
[junit4] 2> 644535 INFO (qtp1833868297-5709) [n:127.0.0.1:36001_solr c:deletereplica_test s:shard1 r:core_node4 x:deletereplica_test_shard1_replica_n3] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 644535 INFO (qtp1833868297-5709) [n:127.0.0.1:36001_solr c:deletereplica_test s:shard1 r:core_node4 x:deletereplica_test_shard1_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'deletereplica_test_shard1_replica_n3' using configuration from collection deletereplica_test, trusted=true
[junit4] 2> 644536 INFO (qtp1833868297-5709) [n:127.0.0.1:36001_solr c:deletereplica_test s:shard1 r:core_node4 x:deletereplica_test_shard1_replica_n3] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36001.solr.core.deletereplica_test.shard1.replica_n3' (registry 'solr.core.deletereplica_test.shard1.replica_n3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@578f7301
[junit4] 2> 644536 INFO (qtp1833868297-5709) [n:127.0.0.1:36001_solr c:deletereplica_test s:shard1 r:core_node4 x:deletereplica_test_shard1_replica_n3] o.a.s.c.SolrCore [[deletereplica_test_shard1_replica_n3] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest_30E77759D5165AE3-001/tempDir-001/node1/deletereplica_test_shard1_replica_n3], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest_30E77759D5165AE3-001/tempDir-001/node1/./deletereplica_test_shard1_replica_n3/data/]
[junit4] 2> 644667 INFO (qtp1833868297-5709) [n:127.0.0.1:36001_solr c:deletereplica_test s:shard1 r:core_node4 x:deletereplica_test_shard1_replica_n3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 644668 INFO (qtp1833868297-5709) [n:127.0.0.1:36001_solr c:deletereplica_test s:shard1 r:core_node4 x:deletereplica_test_shard1_replica_n3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 644669 INFO (qtp1833868297-5709) [n:127.0.0.1:36001_solr c:deletereplica_test s:shard1 r:core_node4 x:deletereplica_test_shard1_replica_n3] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 644669 INFO (qtp1833868297-5709) [n:127.0.0.1:36001_solr c:deletereplica_test s:shard1 r:core_node4 x:deletereplica_test_shard1_replica_n3] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 644671 INFO (qtp1833868297-5709) [n:127.0.0.1:36001_solr c:deletereplica_test s:shard1 r:core_node4 x:deletereplica_test_shard1_replica_n3] o.a.s.s.SolrIndexSearcher Opening [Searcher@16edcf00[deletereplica_test_shard1_replica_n3] main]
[junit4] 2> 644672 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 644672 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 644673 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 644673 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 644675 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2a4c92cc[deletereplica_test_shard1_replica_n1] main]
[junit4] 2> 644676 INFO (qtp1833868297-5709) [n:127.0.0.1:36001_solr c:deletereplica_test s:shard1 r:core_node4 x:deletereplica_test_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 644680 INFO (qtp1833868297-5709) [n:127.0.0.1:36001_solr c:deletereplica_test s:shard1 r:core_node4 x:deletereplica_test_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 644681 INFO (qtp1833868297-5709) [n:127.0.0.1:36001_solr c:deletereplica_test s:shard1 r:core_node4 x:deletereplica_test_shard1_replica_n3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 644681 INFO (qtp1833868297-5709) [n:127.0.0.1:36001_solr c:deletereplica_test s:shard1 r:core_node4 x:deletereplica_test_shard1_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1617934162549800960
[junit4] 2> 644684 INFO (searcherExecutor-2701-thread-1-processing-n:127.0.0.1:36001_solr x:deletereplica_test_shard1_replica_n3 c:deletereplica_test s:shard1 r:core_node4) [n:127.0.0.1:36001_solr c:deletereplica_test s:shard1 r:core_node4 x:deletereplica_test_shard1_replica_n3] o.a.s.c.SolrCore [deletereplica_test_shard1_replica_n3] Registered new searcher Searcher@16edcf00[deletereplica_test_shard1_replica_n3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 644684 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 644684 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 644685 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 644685 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1617934162553995264
[junit4] 2> 644695 INFO (searcherExecutor-2700-thread-1-processing-n:127.0.0.1:39242_solr x:deletereplica_test_shard1_replica_n1 c:deletereplica_test s:shard1 r:core_node2) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.c.SolrCore [deletereplica_test_shard1_replica_n1] Registered new searcher Searcher@2a4c92cc[deletereplica_test_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 644697 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.c.ZkShardTerms Failed to save terms, version is not a match, retrying
[junit4] 2> 644697 INFO (qtp1833868297-5709) [n:127.0.0.1:36001_solr c:deletereplica_test s:shard1 r:core_node4 x:deletereplica_test_shard1_replica_n3] o.a.s.c.ZkShardTerms Successful update of terms at /collections/deletereplica_test/terms/shard1 to Terms{values={core_node4=0}, version=0}
[junit4] 2> 644699 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/deletereplica_test/terms/shard1 to Terms{values={core_node2=0, core_node4=0}, version=1}
[junit4] 2> 644706 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 644706 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 644706 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39242/solr/deletereplica_test_shard1_replica_n1/
[junit4] 2> 644706 DEBUG (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"deletereplica_test"} current state version: 0
[junit4] 2> 644707 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=deletereplica_test_shard1_replica_n1 url=http://127.0.0.1:39242/solr START replicas=[http://127.0.0.1:36001/solr/deletereplica_test_shard1_replica_n3/] nUpdates=100
[junit4] 2> 644711 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=deletereplica_test_shard1_replica_n1 url=http://127.0.0.1:39242/solr DONE. We have no versions. sync failed.
[junit4] 2> 644719 INFO (qtp1833868297-5714) [n:127.0.0.1:36001_solr c:deletereplica_test s:shard1 r:core_node4 x:deletereplica_test_shard1_replica_n3] o.a.s.c.S.Request [deletereplica_test_shard1_replica_n3] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 644719 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 644719 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 644731 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39242/solr/deletereplica_test_shard1_replica_n1/ shard1
[junit4] 2> 644732 DEBUG (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"deletereplica_test",
[junit4] 2> "base_url":"http://127.0.0.1:39242/solr",
[junit4] 2> "core":"deletereplica_test_shard1_replica_n1",
[junit4] 2> "state":"active"} current state version: 0
[junit4] 2> 644836 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 644837 DEBUG (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":"deletereplica_test_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:39242/solr",
[junit4] 2> "node_name":"127.0.0.1:39242_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"deletereplica_test",
[junit4] 2> "type":"NRT",
[junit4] 2> "force_set_state":"false",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 644843 INFO (qtp639475210-5710) [n:127.0.0.1:39242_solr c:deletereplica_test s:shard1 r:core_node2 x:deletereplica_test_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=deletereplica_test_shard1_replica_n1&action=CREATE&numShards=1&collection=deletereplica_test&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1410
[junit4] 2> 644941 INFO (zkCallback-1561-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deletereplica_test/state.json] for collection [deletereplica_test] has occurred - updating... (live nodes size: [4])
[junit4] 2> 645708 DEBUG (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":"deletereplica_test_shard1_replica_n3",
[junit4] 2> "core_node_name":"core_node4",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:36001/solr",
[junit4] 2> "node_name":"127.0.0.1:36001_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"deletereplica_test",
[junit4] 2> "type":"NRT",
[junit4] 2> "force_set_state":"false",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 645709 INFO (qtp1833868297-5709) [n:127.0.0.1:36001_solr c:deletereplica_test s:shard1 r:core_node4 x:deletereplica_test_shard1_replica_n3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=deletereplica_test_shard1_replica_n3&action=CREATE&numShards=1&collection=deletereplica_test&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2264
[junit4] 2> 645719 INFO (qtp1742466670-5720) [n:127.0.0.1:42552_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 645809 INFO (zkCallback-1534-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deletereplica_test/state.json] for collection [deletereplica_test] has occurred - updating... (live nodes size: [4])
[junit4] 2> 645810 INFO (zkCallback-1561-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deletereplica_test/state.json] for collection [deletereplica_test] has occurred - updating... (live nodes size: [4])
[junit4] 2> 646722 INFO (qtp1742466670-5720) [n:127.0.0.1:42552_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf&name=deletereplica_test&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=3697
[junit4] 2> 646729 INFO (qtp639475210-5724) [n:127.0.0.1:39242_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={action=STATUS&indexInfo=true&wt=javabin&version=2} status=0 QTime=3
[junit4] 2> 646731 INFO (qtp1742466670-5716) [n:127.0.0.1:42552_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletereplica with params replica=core_node2&action=DELETEREPLICA&collection=deletereplica_test&shard=shard1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 646740 INFO (OverseerCollectionConfigSetProcessor-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 646745 INFO (qtp639475210-5727) [n:127.0.0.1:39242_solr x:deletereplica_test_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.deletereplica_test.shard1.replica_n1, tag=null
[junit4] 2> 646745 INFO (qtp639475210-5727) [n:127.0.0.1:39242_solr x:deletereplica_test_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4aac674c: rootName = solr_39242, domain = solr.core.deletereplica_test.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.deletereplica_test.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@b1a520e
[junit4] 2> 646774 INFO (qtp639475210-5727) [n:127.0.0.1:39242_solr x:deletereplica_test_shard1_replica_n1] o.a.s.c.SolrCore [deletereplica_test_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@453abec
[junit4] 2> 646774 INFO (qtp639475210-5727) [n:127.0.0.1:39242_solr x:deletereplica_test_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.deletereplica_test.shard1.replica_n1, tag=453abec
[junit4] 2> 646774 INFO (qtp639475210-5727) [n:127.0.0.1:39242_solr x:deletereplica_test_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.deletereplica_test.shard1.leader, tag=453abec
[junit4] 2> 646779 INFO (qtp639475210-5727) [n:127.0.0.1:39242_solr x:deletereplica_test_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/deletereplica_test/terms/shard1 to Terms{values={core_node4=0}, version=2}
[junit4] 2> 646787 INFO (qtp639475210-5727) [n:127.0.0.1:39242_solr x:deletereplica_test_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=deletereplica_test_shard1_replica_n1&deleteMetricsHistory=true&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2&deleteIndex=true} status=0 QTime=42
[junit4] 2> 646788 DEBUG (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"deletecore",
[junit4] 2> "core":"deletereplica_test_shard1_replica_n1",
[junit4] 2> "node_name":"127.0.0.1:39242_solr",
[junit4] 2> "collection":"deletereplica_test",
[junit4] 2> "base_url":"http://127.0.0.1:39242/solr",
[junit4] 2> "core_node_name":"core_node2"} current state version: 0
[junit4] 2> 646789 DEBUG (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"deletereplica_test"} current state version: 0
[junit4] 2> 646789 INFO (zkCallback-1534-thread-2) [n:127.0.0.1:36001_solr c:deletereplica_test s:shard1 r:core_node4 x:deletereplica_test_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 646891 INFO (zkCallback-1561-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deletereplica_test/state.json] for collection [deletereplica_test] has occurred - updating... (live nodes size: [4])
[junit4] 2> 646891 INFO (zkCallback-1534-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deletereplica_test/state.json] for collection [deletereplica_test] has occurred - updating... (live nodes size: [4])
[junit4] 2> 646993 INFO (qtp1742466670-5716) [n:127.0.0.1:42552_solr c:deletereplica_test ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replica=core_node2&action=DELETEREPLICA&collection=deletereplica_test&shard=shard1&wt=javabin&version=2} status=0 QTime=262
[junit4] 2> 648743 INFO (OverseerCollectionConfigSetProcessor-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 649290 INFO (zkCallback-1534-thread-2) [n:127.0.0.1:36001_solr c:deletereplica_test s:shard1 r:core_node4 x:deletereplica_test_shard1_replica_n3] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36001/solr/deletereplica_test_shard1_replica_n3/
[junit4] 2> 649290 INFO (zkCallback-1534-thread-2) [n:127.0.0.1:36001_solr c:deletereplica_test s:shard1 r:core_node4 x:deletereplica_test_shard1_replica_n3] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 649290 INFO (zkCallback-1534-thread-2) [n:127.0.0.1:36001_solr c:deletereplica_test s:shard1 r:core_node4 x:deletereplica_test_shard1_replica_n3] o.a.s.c.SyncStrategy http://127.0.0.1:36001/solr/deletereplica_test_shard1_replica_n3/ has no replicas
[junit4] 2> 649298 DEBUG (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"deletereplica_test",
[junit4] 2> "base_url":"http://127.0.0.1:36001/solr",
[junit4] 2> "core":"deletereplica_test_shard1_replica_n3",
[junit4] 2> "state":"active"} current state version: 0
[junit4] 2> 649310 INFO (zkCallback-1534-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/deletereplica_test/state.json] for collection [deletereplica_test] has occurred - updating... (live nodes size: [4])
[junit4] 2> 649311 INFO (zkCallback-1534-thread-2) [n:127.0.0.1:36001_solr c:deletereplica_test s:shard1 r:core_node4 x:deletereplica_test_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36001/solr/deletereplica_test_shard1_replica_n3/ shard1
[junit4] 2> 649323 INFO (TEST-DeleteReplicaTest.deleteReplicaAndVerifyDirectoryCleanup-seed#[30E77759D5165AE3]) [ ] o.a.s.SolrTestCaseJ4 ###Ending deleteReplicaAndVerifyDirectoryCleanup
[junit4] 2> 649425 INFO (TEST-DeleteReplicaTest.deleteLiveReplicaTest-seed#[30E77759D5165AE3]) [ ] o.a.s.SolrTestCaseJ4 ###Starting deleteLiveReplicaTest
[junit4] 2> 649426 INFO (qtp1742466670-5722) [n:127.0.0.1:42552_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf&name=delLiveColl&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 649432 INFO (OverseerThreadFactory-2677-thread-3-processing-n:127.0.0.1:42552_solr) [n:127.0.0.1:42552_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection delLiveColl
[junit4] 2> 649444 DEBUG (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "name":"delLiveColl",
[junit4] 2> "fromApi":"true",
[junit4] 2> "collection.configName":"conf",
[junit4] 2> "numShards":"2",
[junit4] 2> "nrtReplicas":"2",
[junit4] 2> "stateFormat":"2",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "operation":"create"} current state version: 0
[junit4] 2> 649657 INFO (qtp1742466670-5719) [n:127.0.0.1:42552_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 649664 INFO (qtp54270777-5711) [n:127.0.0.1:43824_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=1
[junit4] 2> 649670 INFO (qtp1833868297-5714) [n:127.0.0.1:36001_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.deletereplica_test.shard1.replica_n3:INDEX.sizeInBytes} status=0 QTime=1
[junit4] 2> 649674 INFO (qtp1833868297-5731) [n:127.0.0.1:36001_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=1
[junit4] 2> 649677 INFO (qtp639475210-5730) [n:127.0.0.1:39242_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 649694 DEBUG (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"delLiveColl",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"delLiveColl_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:39242/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 649694 INFO (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"delLiveColl",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"delLiveColl_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:39242/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 649702 DEBUG (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"delLiveColl",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"delLiveColl_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:43824/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 649702 INFO (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"delLiveColl",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"delLiveColl_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:43824/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 649717 DEBUG (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"delLiveColl",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"delLiveColl_shard2_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:42552/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 649717 INFO (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"delLiveColl",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"delLiveColl_shard2_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:42552/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 649722 DEBUG (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.Overseer processMessage: queueSize: 4, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"delLiveColl",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"delLiveColl_shard2_replica_n6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:42552/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 649722 INFO (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"delLiveColl",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"delLiveColl_shard2_replica_n6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:42552/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 649928 INFO (qtp639475210-5728) [n:127.0.0.1:39242_solr x:delLiveColl_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=delLiveColl_shard1_replica_n1&action=CREATE&numShards=2&collection=delLiveColl&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 649929 INFO (qtp54270777-5708) [n:127.0.0.1:43824_solr x:delLiveColl_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf&newCollection=true&name=delLiveColl_shard1_replica_n2&action=CREATE&numShards=2&collection=delLiveColl&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 649940 DEBUG (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":"delLiveColl_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:39242/solr",
[junit4] 2> "node_name":"127.0.0.1:39242_solr",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"delLiveColl",
[junit4] 2> "type":"NRT",
[junit4] 2> "force_set_state":"false",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 649943 INFO (qtp1742466670-5718) [n:127.0.0.1:42552_solr x:delLiveColl_shard2_replica_n6] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf&newCollection=true&name=delLiveColl_shard2_replica_n6&action=CREATE&numShards=2&collection=delLiveColl&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 649943 INFO (qtp1742466670-5721) [n:127.0.0.1:42552_solr x:delLiveColl_shard2_replica_n4] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf&newCollection=true&name=delLiveColl_shard2_replica_n4&action=CREATE&numShards=2&collection=delLiveColl&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 649944 INFO (qtp1742466670-5721) [n:127.0.0.1:42552_solr x:delLiveColl_shard2_replica_n4] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 649954 DEBUG (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "core":"delLiveColl_shard1_replica_n2",
[junit4] 2> "core_node_name":"core_node5",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:43824/solr",
[junit4] 2> "node_name":"127.0.0.1:43824_solr",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"delLiveColl",
[junit4] 2> "type":"NRT",
[junit4] 2> "force_set_state":"false",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 649959 DEBUG (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.Overseer processMessage: queueSize: 4, message = {
[junit4] 2> "core":"delLiveColl_shard2_replica_n4",
[junit4] 2> "core_node_name":"core_node7",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:42552/solr",
[junit4] 2> "node_name":"127.0.0.1:42552_solr",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"delLiveColl",
[junit4] 2> "type":"NRT",
[junit4] 2> "force_set_state":"false",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 649959 DEBUG (OverseerStateUpdate-72327408122331144-127.0.0.1:42552_solr-n_0000000000) [n:127.0.0.1:42552_solr ] o.a.s.c.Overseer processMessage: queueSize: 4, message = {
[junit4] 2> "core":"delLiveColl_shard2_replica_n6",
[junit4] 2> "core_node_name":"core_node8",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:42552/solr",
[junit4] 2> "node_name":"127.0.0.1:42552_solr",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"delLiveColl",
[junit4] 2> "type":"NRT",
[junit4] 2> "force_set_state":"false",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 650958 INFO (qtp639475210-5728) [n:127.0.0.1:39242_solr c:delLiveColl s:shard1 r:core_node3 x:delLiveColl_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 650973 INFO (qtp54270777-5708) [n:127.0.0.1:43824_solr c:delLiveColl s:shard1 r:core_node5 x:delLiveColl_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 650977 INFO (qtp639475210-5728) [n:127.0.0.1:39242_solr c:delLiveColl s:shard1 r:core_node3 x:delLiveColl_shard1_replica_n1] o.a.s.s.IndexSchema [delLiveColl_shard1_replica_n1] Schema name=minimal
[junit4] 2> 651004 INFO (qtp639475210-5728) [n:127.0.0.1:39242_solr c:delLiveColl s:shard1 r:core_node3 x:delLiveColl_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 651004 INFO (qtp639475210-5728) [n:127.0.0.1:39242_solr c:delLiveColl s:shard1 r:core_node3 x:delLiveColl_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'delLiveColl_shard1_replica_n1' using configuration from collection delLiveColl, trusted=true
[junit4] 2> 651004 INFO (qtp639475210-5728) [n:127.0.0.1:39242_solr c:delLiveColl s:shard1 r:core_node3 x:delLiveColl_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39242.solr.core.delLiveColl.shard1.replica_n1' (registry 'solr.core.delLiveColl.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@578f7301
[junit4] 2> 651005 INFO (qtp639475210-5728) [n:127.0.0.1:39242_solr c:delLiveColl s:shard1 r:core_node3 x:delLiveColl_shard1_replica_n1] o.a.s.c.SolrCore [[delLiveColl_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest_30E77759D5165AE3-001/tempDir-001/node3/delLiveColl_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest_30E77759D5165AE3-001/tempDir-001/node3/./delLiveColl_shard1_replica_n1/data/]
[junit4] 2> 651011 INFO (qtp1742466670-5721) [n:127.0.0.1:42552_solr c:delLiveColl s:shard2 r:core_node7 x:delLiveColl_shard2_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 651012 INFO (qtp1742466670-5718) [n:127.0.0.1:42552_solr c:delLiveColl s:shard2 r:core_node8 x:delLiveColl_shard2_replica_n6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 651019 INFO (qtp54270777-5708) [n:127.0.0.1:43824_solr c:delLiveColl s:shard1 r:core_node5 x:delLiveColl_shard1_replica_n2] o.a.s.s.IndexSchema [delLiveColl_shard1_replica_n2] Schema name=minimal
[junit4] 2> 651022 INFO (qtp54270777-5708) [n:127.0.0.1:43824_solr c:delLiveColl s:shard1 r:core_node5 x:delLiveColl
[...truncated too long message...]
ere are still nodes recoverying - waited for 320 seconds
[junit4] > at __randomizedtesting.SeedInfo.seed([30E77759D5165AE3:B8B348837BEA371B]:0)
[junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:185)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:920)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForThingsToLevelOut(AbstractFullDistribZkTestBase.java:1477)
[junit4] > at org.apache.solr.cloud.RestartWhileUpdatingTest.test(RestartWhileUpdatingTest.java:145)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1010)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:985)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 3113369 WARN (SUITE-HdfsRestartWhileUpdatingTest-seed#[30E77759D5165AE3]-worker) [ ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
[junit4] 2> 3113385 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[30E77759D5165AE3]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 3113487 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_30E77759D5165AE3-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_30E77759D5165AE3-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to localhost/127.0.0.1:38117) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 3113487 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_30E77759D5165AE3-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_30E77759D5165AE3-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to localhost/127.0.0.1:38117) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-1895730324-10.41.0.5-1542984336112 (Datanode Uuid 3d82ee50-b07d-47e6-b0b8-74c1ebe97c4a) service to localhost/127.0.0.1:38117
[junit4] 2> 3113493 WARN (SUITE-HdfsRestartWhileUpdatingTest-seed#[30E77759D5165AE3]-worker) [ ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
[junit4] 2> 3113507 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[30E77759D5165AE3]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 3113611 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_30E77759D5165AE3-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_30E77759D5165AE3-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:38117) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 3113611 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_30E77759D5165AE3-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_30E77759D5165AE3-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:38117) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-1895730324-10.41.0.5-1542984336112 (Datanode Uuid 63a6b2b6-751c-4907-a0b4-f5e3318adcf1) service to localhost/127.0.0.1:38117
[junit4] 2> 3113750 INFO (SUITE-HdfsRestartWhileUpdatingTest-seed#[30E77759D5165AE3]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRestartWhileUpdatingTest_30E77759D5165AE3-001
[junit4] 2> Nov 23, 2018 2:52:05 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 32 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene80), sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@6a444416), locale=und, timezone=Africa/Bujumbura
[junit4] 2> NOTE: Linux 4.4.0-137-generic amd64/Oracle Corporation 1.8.0_191 (64-bit)/cpus=4,threads=3,free=56745456,total=522190848
[junit4] 2> NOTE: All tests run in this JVM: [CloneFieldUpdateProcessorFactoryTest, TestRequestForwarding, DistributedQueryComponentOptimizationTest, AddBlockUpdateTest, ZkStateWriterTest, ChaosMonkeyNothingIsSafeTest, ClassificationUpdateProcessorFactoryTest, DistributedVersionInfoTest, CollectionPropsTest, TestHalfAndHalfDocValues, WordBreakSolrSpellCheckerTest, TestCryptoKeys, ConcurrentCreateRoutedAliasTest, MinimalSchemaTest, CdcrWithNodesRestartsTest, TestFieldTypeResource, TestNumericTerms32, AddSchemaFieldsUpdateProcessorFactoryTest, DocumentAnalysisRequestHandlerTest, TestSweetSpotSimilarityFactory, BufferStoreTest, TestApiFramework, TestSolrDeletionPolicy1, MetricsHistoryIntegrationTest, TestFunctionQuery, OverseerTaskQueueTest, RestartWhileUpdatingTest, HLLSerializationTest, SearchRateTriggerIntegrationTest, TestDistributedSearch, TestComplexPhraseLeadingWildcard, DOMUtilTest, DisMaxRequestHandlerTest, TestDocBasedVersionConstraints, SolrIndexMetricsTest, ReplaceNodeNoTargetTest, BooleanFieldTest, EmbeddedSolrNoSerializeTest, GraphQueryTest, SolrInfoBeanTest, FileUtilsTest, XmlUpdateRequestHandlerTest, HdfsNNFailoverTest, TestSimPolicyCloud, TestSolr4Spatial2, ShufflingReplicaListTransformerTest, TestLuceneMatchVersion, TestLegacyNumericRangeQueryBuilder, HdfsTlogReplayBufferedWhileIndexingTest, SpellPossibilityIteratorTest, SuggesterTest, TestHighlightDedupGrouping, TaggerTest, RegexBoostProcessorTest, TestManagedSchemaThreadSafety, CollectionTooManyReplicasTest, AutoAddReplicasPlanActionTest, TestPHPSerializedResponseWriter, PreAnalyzedFieldManagedSchemaCloudTest, TestSolrCoreSnapshots, MoreLikeThisHandlerTest, TestReplicaProperties, TestInPlaceUpdatesDistrib, TestUnInvertedFieldException, TestMultiValuedNumericRangeQuery, RecoveryAfterSoftCommitTest, DeleteShardTest, ExitableDirectoryReaderTest, MBeansHandlerTest, RulesTest, TestMacros, PKIAuthenticationIntegrationTest, TestCollectionsAPIViaSolrCloudCluster, TestJsonFacetsStatsParsing, SolrRrdBackendFactoryTest, TestSolrCloudWithDelegationTokens, CdcrBidirectionalTest, TimeZoneUtilsTest, ShardSplitTest, TestRTGBase, PropertiesRequestHandlerTest, TestSimpleQParserPlugin, VMParamsZkACLAndCredentialsProvidersTest, ConcurrentDeleteAndCreateCollectionTest, TokenizerChainTest, PluginInfoTest, TriggerIntegrationTest, TestPhraseSuggestions, TestMissingGroups, CheckHdfsIndexTest, TestXmlQParser, TestStressCloudBlindAtomicUpdates, EnumFieldTest, PrimUtilsTest, TestRandomFlRTGCloud, TestCopyFieldCollectionResource, TestSolrXml, MetricUtilsTest, HighlighterTest, TestCloudDeleteByQuery, CursorMarkTest, TestSystemCollAutoCreate, TestCloudRecovery2, NumericFieldsTest, AliasIntegrationTest, TestInfoStreamLogging, TestHashPartitioner, ExecutePlanActionTest, TestQuerySenderListener, NoCacheHeaderTest, TestNumericTokenStream, TestPrepRecovery, TestSlowCompositeReaderWrapper, CdcrReplicationHandlerTest, SolrCoreCheckLockOnStartupTest, TestFastOutputStream, TestStressReorder, DistributedFacetPivotLongTailTest, TestTrackingShardHandlerFactory, MaxSizeAutoCommitTest, TestIntervalFaceting, TestOverriddenPrefixQueryForCustomFieldType, DeleteLastCustomShardedReplicaTest, LeaderElectionTest, TestSurroundQueryParser, HdfsRestartWhileUpdatingTest]
[junit4] Completed [484/836 (3!)] on J1 in 393.28s, 1 test, 1 failure <<< FAILURES!
[...truncated 1256 lines...]
[junit4] JVM J2: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/temp/junit4-J2-20181123_140011_6402395648182180619384.sysout
[junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
[junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/heapdumps/java_pid16161.hprof ...
[junit4] Heap dump file created [583773827 bytes in 4.836 secs]
[junit4] <<< JVM J2: EOF ----
[...truncated 9024 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:651: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:585: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid16161.hprof
Total time: 320 minutes 57 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any